I bought a beautiful old, low time, straight tail 172 for $25,000, scratched my nuts, and mumbled "hmmm, guess I ought to learn how to fly, huh?"
I learned to fly in it and put over 250 hours on it. Then I found a beautiful old, low time, straight tail 182. I bought it, and sold the 172 for the same $25,000 that I paid for it. Learning to fly in my own plane was the bomb!! I wouldn't trade that experience for anything...especially since I'm overly anal retentive about both proper mx and cleanliness.
I used to beat myself up over how many hours I had when I finally went to take my check ride. Well over 100. I thought I must be really stupid. Then one day I analyzed how many of those hours were instructional and how many were just me going out and playing after I soloed. 48 were instructional and over 60 were me just playing since I owned a plane and mentally all that the flying was costing me was gas. I also had access to 4 airports that were within the magic 24nm distance and that I'd been signed off to fly repeat XCs to. So I'd fly round robin XCs, do two or threelandings at each, or just go out and play and practice upper air work. I would have never done that had I been paying full rental price.
One other benefit of owning, if you're in a region where 0E autogas is available, and you buy a plane that can be, or is, STC'ed for MoGas, then you can save some substantial money on fuel. MoGas can run $1 to $2 cheaper than AvGas at the field in many places. The price difference when I was training was typically in the $1.50 range and I'd try to run a blend between 2/1 and 3/1 MoGas/AvGas. This alone saved over $2,000.
I think that depends greatly on what you know, who you know, and where you live. My transacational costs were next to nil except for sales tax.
The key is finding a good plane and, like others have said, make sure you are committed to see it through before you buy.
Or, maybe I just got lucky..........