They don't. Period. it's that simple. Have you ever -- even one time -- seen or heard anything, anywhere other than the FBO bulletin board, in the way of advertising for either CB or Millard? Nope. You're not likely to, either.
The average non-pilot knows absolutely nothing about flying, other than they can buy a ticket on an airliner to go wherever they want to go (as long as it's a major city). Those guys flying around in those scary dangerous little planes? They're all rich guys, or crazy grizzled old Vietnam vets crop dusters who are probably smuggling dope on the side. The average person thinks a Warrior or 172 probably costs more than their house.
There are exceptions. My wife's family are a little more familiar with GA, because one of her farmer uncles had a plane back in the 70s. Still, there's the perception that it's for rich people (Uncle Farmer was pretty well off back when there was a highly speculative hog market). I'm trying to spread the word in my family and among our friends, but flying is regarded as about two notches past motorcycle riding on the crazy scale. And guys who think nothing of blowing half a month's mortgage on a golf weekend still think flying is for rich guys.
A couple of people have remarked about the age thing. I just turned 53, and I'm one of the youngest people to show up at EAA and flying club meetings. Much less so the flying club, though; we are seeing some younger blood there because it's a great deal if you fly more than a few hours a month. Still, I know I advertise the club (I'm membership coordinator) more than Hangar One advertises flight school, and all I've done is post an ad on Craigslist when there was a membership for sale.
I am encouraged by the fact that the student pilots seem to be relatively younger (20s and 30s), and there seem to be quite a few of them. Millard usually has at least one or two Cherokees in the pattern on a decent day, and you putter around in or near the practice area at your own peril. Still, there's room for a lot more flying and a lot more instructing.
Root cause? I don't know. Decades of economic beat-down. Decades of risk-averse "You'll shoot your eye out, kid". The complete collapse of the US airplane manufacturing industry in the 80s. An advertising-driven, consumer mindset seems to keep most people unaware of or uninterested in anything other than "pay someone to do this for me". Take your pick, there's plenty of reasons to go around. There's only one thing to do about it...
Go fly. And invite someone to go along with you!