I think we're talking past each other a bit - I was talking abut accident "rates", for which we have no data of a reliable nature - we can count mid-airs, for example, which are very rare, I definitely concede, but we don't know how many per flying hour are happening, anywhere, 'cause we don't know how many hours are being flown, anywhere. GA hours are a guess. We'd need a time interval to get a rate, so we can't give the OP hard facts.
Admitting that I'm just SWAG'ing myself, I've experienced opposite landing traffic on runways and in the pattern, runway incursions, folks entering downwind with me bore sighted, etc., at a higher perceived "rate" than at towered fields or deserted uncontrolled fields on a Monday morning. Pretty sure those events made a bad outcome a bit more likely, but, admittedly, not inevitable.
If you're saying the OP is more likely to get hurt in a loss of control than a mid-air, I concur; I was focusing on things other than stick-and-rudder skills, over which the OP has some, but less, control. Pressing weather that's coming down, or pressing fuel margins, or mixing it up with yahoo's at an uncontrolled field.
My advice would still be to stay out of IMC, don't out fly your fuel, and avoid the Battle Of Britain patterns at uncontrolled fields. You could add "don't land long" and be x-wind proficient; I think most of those kinds of incidents aren't fatal, though, at least not in the lower end of the GA fleet.