I've been kind of curious while learning for PPL what makes an acrobatic aircraft be able to do loops, etc.
Technically all aircraft are three-dimensional... if you didn't put more than the certified stresses on the airframe, they can all do basic "aerobatics"... FAA doesn't look too kindly on it in things that aren't at least certified to the Utility Category for certain maneuvers and of course, higher for other stuff, but the airplanes don't really care. For the reasons behind why some can do it better than others, that's about the loads presented by centrifugal and centripetal forces.
Think "steep turn". Gravity didn't go up, but you're putting 2Gs of load on the airframe in a 60 degree banked steep turn. That's a force added to gravity, but it's not gravity. Get set up in a level 2G turn and then pull real hard, and it'll be more than 2Gs to climb from it... you get the idea.
I'm just wondering. Assuming maybe wings design, that it can take more g force, but when a plane does a loop, does it pass through a stall, and then sort of go over to inverted flight?
Is it the thrust (like a helicopter) of the propeller, size of the motor that gives enough thrust?
Wing design, wing attach points, physical structure strength, bigger engines, yes... all of that. Bigger flight control surfaces, some aircraft have "spades" attached to the ailerons to help with stick forces... etc etc etc...
Some "aerobatic" airplanes are better at it than others, too. Everything is a trade-off.
And no, no real reason to stall to do a loop, just very little/no "lift" in the downward direction (toward earth) at the top.
Just what is going on aerodynamically in some of these amazing maneuvers?
Technically the aerodynamics isn't different from any other time... maybe during inverted flight since the wing is kinda doing its thing back-assward, but not much true aerodynamic difference. For you sitting in the seat, you're probably not used to being hung from your toes upside down, so that'll FEEL quite odd, but the airplane doesn't really care, aerodynamically.
It DOES care for things like oil sumps and pumps and all the usual mechanical stuff that normally occurs upright... many "aerobatic" aircraft have inverted-oil systems for the engine, for example... some don't and have limits on how long you can be without our old pal gravity in the usual downward/toward the landing gear direction.
You COULD barrel roll a Skyhawk AND keep it well within the load limits of the aircraft, if you knew what you were doing... but it's very ill-advised (and illegal) because it's easy to screw it up and overstress the structure of the airframe. Make sense? It's not really the rules or the certification that makes an airplane "aerobatic", they really make the airplane "much more suitable for aerobatic flight", truly.