I would argue that both segments of aviation have their unique challenges. I got my PPL in High School and now fly F/A-18s. Couple points;
1. I landed on an aircraft carrier with 190 hours of military time. In the civil world I wouldn't be allowed to get paid for a cross country...yet I got in a jet solo, flew 100 miles off the coast and landed on a ship. No way to compare that amount of trust and responsibility to civil aviation.
2. In the military/tactical world problems happen much faster. Flying a low level at 200 feet/480 knots requires much faster thinking and reaction time than anything in the civil world. Period dot.
3. The military teaches to fly the aircraft to the edges of its performance envelope. We spun the T-6 right side up, left, right, out of a turn, level etc. I hadn't done any supervised spins in civil flight training and only did a couple on my own in a Super Decathlon. Civil aviation teaches people to avoid max performing their aircraft if possible.
4. The biggest challenges of transport aviation, taking off, flying a complex route to a challenging approach to mins is the same thing we need to do in the military...after we have completed a mission, employed complex guided ordnance, and done it all in a plane that carries nowhere near enough fuel and usually have much less advanced cockpit instrumentation compared to our civil counterparts.
5. That being said, civil aviation offers it's own challenges, off field landings, uncontrolled ops, flying non-standard aircraft...etc. It's possible to die in either job. Neither is necessarily "better" but to say that the military doesn't require more from its aviators than civilian ops is naive. In military ops, flying is just admin.
6. The problem with civil aviation is lack of QA. I've seen some wonderful CFIs, but also lots of slackers and unprofessional individuals...in the military quality is much more standardized. Fewer rock stars, but much fewer no-loads too. Also, I took a checkride about twice a month in the training command, every flight was an evaluation that could be failed. That provides pressure to learn and take onboard lessons quick, vice a civil environment where a CFI may not provide a harsh debrief for fear or losing a student and not being able to make rent that month.
All that being said, GA holds a dear place in my heart and is just as much fun as my day job...