I appreciate that sentiment of getting to specifics.
In my case though, I think of Vx NOT so much in terms of the "u-turn"...but instead in terms of the straight ahead landing....just for that very low altitude portion of the departure leg when no matter the climb rate you aren't going to be high enough to glide to any of the options outside the airport boundary.
Again, yeah I get it....granted this introduces more risk because of the high AOA and need for no delay push over
But just as an exaggerated thought experiment...
1) If I can be at TPA by the time I'm at the runway's mid point and at that point my engine shuts down cold, I can see three obvious options
a) slip to a landing straight ahead
b) circle back downwind to land on the same runway same direction
c) I might be high enough to glide lots of other places both on and off the airport
(I'm thinking a little Cub or a Champ or any of those STOL competition aircraft you see at Sun n Fun, etc...)
2) On the other hand.... climbing at what might seem like a cruise speed, very low climb rate...I'm at half TPA well outside the airport fence. Now what are my options, 500 ft AGL out over anything but open spaces? That glide ring is pretty small....
(I'm currently thinking of the climb rate I experienced in multi training in that old tired Apache 150, climb gradient nearly flat even with two engines turning, with one...yeah that was eye opening)