Lousy low-altitude maneuverability at lower speeds. Corner velocity was about 440 knots, so hard maneuvering at lower speeds resulted in a big, fast loss of energy, and keeping the speed higher than that a) meant running in burner all the time, which meant about 20 minutes to dry tanks, and b) spreading the show over too large an area (both for containment and for spectator viewing).
...which is why it was called the "Double Ugly" or "Rhino" in its day.
Primarily fuel cost -- one F100 engine runs a lot cheaper than two.