Wow. I think I'm your twin.
I just got my PPL this year and balanced the expense by selling a motorcycle. I'm hoping to squeeze some flying into the ongoing family budget but your experience mirrors mine. 80th percentile isn't enough to afford flying these days unless your entire family enjoys it as much as you do.
Ditto. I'm fairly far along in my training, but going into it I had the mindset that the fun would be in the journey, not the destination. If I got 30 hours in and decided to quit, as long as I enjoyed the life experience to that point I wouldn't consider it a waste. I waited to start until I had the discretionary money to spend - sure we could have used it in other ways, and to an extent the rest of the family had to make some sacrifices to make it possible. However, I think that my wife will enjoy flying as well, and from time to time I try to bring her along to a meetup/etc and pay for her to go on a discovery flight/etc so that she isn't entirely left out.
I'm enjoying the experience of everything, but I still end up wondering from time to time if it is a worthwhile investment. I keep trying to look at it the way I'd look at dining out or going on a vacation - you don't do this stuff with the goal of having something to show for it, but for the experience. Down the road if my financial situation changes I'd need to re-consider things, but at the moment I'm between the raising kids phase and the retirement crunch, so I might as well get the experience in.
Once I have my license I'm sure my wife and I will go flying around/etc, and we'll end up spending $350 on trips that could cost us $50, but that's OK as long as we get what we want out of it.
For me it has also been a bit of a personal journey as well. Honestly, I was probably white-knuckled on my discovery flight. I flew on a flight simulator for the first time when I was in elementary school (on a C64 I think) and have been flying them somewhat regularly ever since. At the same time, I've always tended to have a firm grip on the armrest during landing even in a 747, and that was nothing compared to my discovery flight in a 172. Even 30 hours in I had a flight the other week with crosswinds at ~8G15 that I ended up calling short - it was good practice but the anxiety level was pretty high. It was a good experience all the same as I was able to stay in control of things for the most part - certainly not my nicest approaches but for the most part I had positive control of the plane. I probably won't be soloing in those conditions for a while longer, but I'm getting there. For me part of this is about reconciling my enjoyment of the simulation of flying with my anxiety about sharing control of a plane with the winds and other unseen influences on the movement of the surrounding air. Sure, for what it costs to fly around in a 1974 172M I could be flying a pretty decent simulation of a business jet in my basement, but there is something nice about actually going somewhere in the process.
Cost is still likely to be the limiting factor to this as a hobby - more than any other hobby I have aviation just burns through the cash. Not a lot of people have that kind of disposable income lying around - it is a bit of a stretch for me (fairly well-along in an IT career), and it certainly doesn't hurt that I have an understanding wife who enjoys flying (and is far more comfortable with it).
So who knows - maybe I will end up just building a better cockpit with nicer avionics in my basement on a lower budget all the same. However, I think my life and my family's will have been richer for the experience of general aviation, and that's really my goal.