I kinda think if you permitted under an private pilot certificate with IR and a couple CFI endorsements to strap on a 6000# airplane with 5 passengers on board doing 325 knots true in IMC at night an initial class 3 exam isn’t overly burdensome.
I kinda think it is.
Consider that with no medical exam at all you can load a dozen or so passengers into your poorly maintained 1980 van and sail down the most dangerous stretch of interstate in the country, in heavy traffic, in the rain, having last taken a driving test when you were in high school.
Society has decided that risk is acceptable.
The pilot you’re so worried about has had a medical exam, passed a course in aeromedical risk management, had a flight review within the last two years, has made at least three successful landings in the last 90 days, and is flying a plane which has had an annual inspection to ascertain its airworthiness.