The answer is usually, whenever is most inconvenient for a vacation or trip you have scheduled. So finding a good mechanic that you trust, can help work with your schedule and is willing to fix when appropriate rather than just replace is essential.
Financial sense is only part of it. Some years you will end up with an hourly operating cost that comes out ahead of renting, some years not. Of course you carry the risk on maintenance, and each annual on a 40 year old plane is a bit of a lottery.
The upside includes availability and flexibility of the plane, improved safety (you know the plane and it's history intimately), and that pride and enjoyment of ownership (see note above about the "annual lottery"). The flexibility does have some financial value to it, as most flight schools require a minimum daily rental if you're staying overnight, limiting the ability to take, or increasing the cost of longer trips
A good pre-buy is essential. It's been said above but I'll say it again, that's not the same as an annual. It's an inspection with a view to creating a list of items that should be address before sale, soon after, at the next annual and things to watch for the future. Best conducted by a mechanic familiar with the type, and most type clubs can put you in touch with one of those.
Sounds like your girls are at a similar age to mine, and I held back for a while for the same reason. The potential unknown maintenance cost risk still looms, but the fun we are having as a family together with it makes it worthwhile, in my view.
My annual is due in September, perhaps I should repost on October 1 and see if I still feel the same way!