Here's the thing - the stress (read "fear") is cumulative, like radiation exposure. Over a long enough period, and with enough close calls or rough experiences, your subconscious starts raising hell with you for putting it in those situations. A USAF medico (medical or psych, I can't remember) wrote a book about it.
You get that "anyplace else but here" effect.
Shorter term, after a big scare, the old "get back on the horse" approach works pretty well. Or, after a brief(ish) pause, let that near-trauma fade a bit, and you're good to go. Like we tell students to persevere, and not pack it in too soon.
Some satisfaction and self actualization springs from success (handling the x-wind, turbulence, etc.), but the "out of my control" events take a bigger toll - like near-misses, real partial panel in tough IFR, being a witness to a fatal, etc.
It ain't age so much as experience, though there is clearly correlation.