Being part of the working underclass (non-capital owners) means we have either/or choices that limit our discretionary expenditures. Affluent people don't have to limit their discretion.
This isn't really true. I know a number of "affluent" people who have very significant limitations on what they can and can't do with their money, and even more so, their time. They're so valuable to their own businesses that they often have significant insurance policies they must pay for, in order to maintain business continuity if they're suddenly seriously injured or killed, etc. Their "stuff" requires maintenance and operational costs are high on many of the "good life" things they own. It all looks shiny and happy from the outside but it often isn't for them.
Most have developed a keen sense of whether or not they can "blow" money on something that won't further their business interests, or that will negatively affect the ability of their investments and things they own, to pay for those things, and "toys" are only bought when there would be no possible impact to their overall budget.
And let's not forget the mental toll. I know two multi-millionaires (well one was nearly wiped out recently by a trusted person hell-bent on destroying what he had built from the inside) who have disastrous relationships with family including spouses and ex-spouses that make their lives very difficult. Like some have pointed out, their relationships with their kids are tough at best, non-existent at worst because of the hours they keep.
One struggles with being bipolar, mostly as a result of both being extremely poor as a child and also from physical and mental abuse. His "payback" is that he played to his strengths and when he's in "manic" mode, there's no better person at making money at his business (construction of high end homes and remodeling in SoCal), but when he's in "depressive" mode he's a holy terror to himself, his family, and all of his staff. The staff stay because of the good times. He knows he's insufferable at times and makes good on it by paying better than average and showering his staff with "stuff" they want when he's in the good half of his brain. His family mostly doesn't talk to him anymore, because of the unpredictable nature of his behavior. The money he makes is his way of flipping off his childhood but he's permanently messed up from it. By all outward appearances he's hugely "successful", but it's never quite that simple in real life.
Another is very well adjusted but has a funny quirk. He's a vegetarian. One time we asked him what made him decide to be a vegetarian and the answer was shocking to his friends present. He said back when his second kid was born and he was struggling to get his business started, he and his wife wanted to eat healthy but couldn't afford meat. He got used to it over the next few years and never felt a need to go back to eating even cheap hamburger later on. His business before he semi-retired is worth something like $50M I think. Could be way higher, but not much lower. It's not publicly traded so I've had to take a guess over the years. Just their inventory would probably be up around $20M. It operates in multiple states and is smaller than it once was, but more focused on a very high end clientele now.
He spends most of his time in "retirement" teaching fiscal stuff to high schoolers through business clubs and teaching people what he knows about electronics. He drives a tasteful but not flashy "luxury car" that's a few years old, and is one of the nicest people you'd care to meet. If his "thing" was airplanes he would probably buy a really nice one and that'd be his one big splurge.
That how he is with his hobby, he has really nice gear for it and that's it. His house is a little bigger than some, in a nice area, and he could afford a lot more. When you see him at local events for that hobby, you'd never know he was worth millions if you didn't know the business he founded. It was a household name in retail around here when I was a kid. You'd hear radio ads for it and all that. Nowadays it's still around but has a lower profile. Kinda like him. He could have squeezed the lemon harder, but instead he pays well, offers full bennies in a business that rarely does that, including 401K, and medical, and it attracts some top notch folk to his company. Smart guy. Hard to believe he once couldn't afford hamburgers for dinner, but he couldn't.