Reported impressive reduction in accidents due to ADS-B In

That, and I am cursing, is ****ING AWESOME. The planes that didn't hit each other can be so happy with their stupid "safety" device.

IBTL
 
Possible self-selecting dataset?
 
Not sure how adsb helps on takeoffs and landings
 
Where are the p-values?

I'm guessing they would not pass muster for statistical significance.

It looks like the average proportion of ADS-B in equipage during the 5-year study period was about 5% while the average number of mid-airs per year was 0 for equipped and about 10 for non-equipped.

Some simple math, 5% (equipage rate) of 10 (number of non-equipped mid-airs) is less than 1 so how could you possibly draw a conclusion about whether or not ADS-B reduces mid-airs?
 
Last edited:
kyleb said:
Let me be the first to call bullship on this study.
Click to expand...

I am skeptical of anything AOPA publishes. From my own perspective though, I think it has drastically increased airborne safety. Before, we never knew how close we really were. Now, I watch them paint and look out the window, continuously referencing back to the target and ill be damned.,.... there it is, under a mile away before I can see it.

I'm curious what you are calling BS on.
 
I'd have to look carefully at the methodology to see if the study compared similar cohorts of aircraft or, as I suspect, just looked at the entire fleet and ascribed causality to an observed correlation that may be fraught with covariance with other factors like aircraft capability or operator experience.

Just so you know, the FAA is using this study to justify increased funding for additional ground stations which is probably a good thing for all of us even if it's based on naive data analysis.

On the face of it it does seem a little fantastical that installing ADS-B reduces the accident rate by 53%, especially considering much of the fleet is already equipped with portable ADS-B in. Those aircraft should have seen similar safety gains.
 
I found it odd that in table 2, weather related accidents took a big jump in 2017 (some in 2016 too) even with the ADS-B equipped planes.

Yes, I do know that is a large reduction from non-equipped, but wondering why the increase all of a sudden for the equipped planes.
 
I love ADS-B, there I said it. It is great to have and really enhances traffic awareness, not to mention weather awareness. The actual study, posted above looks pretty thorough at first glance. This is great news if it holds up.
 
dmspilot said:
Where are the p-values?
Click to expand...

Appears they used a binomial test based on estimated total operations and reported accidents. They say they only report statistically significant results but don’t give details on the p-level (presumably 5%) or how they corrected for multiple comparisons.

Nonetheless, on first read and without the details that seems as though it is valid. The effect size here is large (surprisingly so actually) and they have estimated fairly large number of operations, so a significant effect would not be surprising.

Could be other methodological issues? It seems like a very large effect.
 
asicer said:
Possible self-selecting dataset?
Click to expand...

I think this is the potential big confound. Maybe people who took the trouble to have ADS-B during this timeframe are also more conscientious pilots.

Seems like that could be addressed in the dataset because the NTSB reports usually have estimates of pilot experience and currency.
 
PeterNSteinmetz said:
I think this is the potential big confound. Maybe people who took the trouble to have ADS-B during this timeframe are also more conscientious pilots.
Click to expand...
Or combined the equipment with other maintenance items like repairing a scratchy radio or replacing a cloudy windshield.
 
Let me help you ADS-B clowns out:

1) Something didn't happen today - Praise the ADS-B system

2) Something happened less frequently in our study period than our made up prior history period - Praise the ADS-B system

3) "I've got ADS-B and I don't like looking out the window" You're families will not praise the ADS-B system, unless you are heavily insured.
 
Ravioli said:
Let me help you ADS-B clowns out:

1) Something didn't happen today - Praise the ADS-B system

2) Something happened less frequently in our study period than our made up prior history period - Praise the ADS-B system

3) "I've got ADS-B and I don't like looking out the window" You're families will not praise the ADS-B system, unless you are heavily insured.
Click to expand...

Why does it have to be black or white Rob? Have you flown in an airplane with traffic reporting?
 
Ravioli said:
Let me help you ADS-B clowns out:

1) Something didn't happen today - Praise the ADS-B system

2) Something happened less frequently in our study period than our made up prior history period - Praise the ADS-B system

3) "I've got ADS-B and I don't like looking out the window" You're families will not praise the ADS-B system, unless you are heavily insured.
Click to expand...

ADS-B is great technology, no doubt. The "free" <ack!> weather and traffic is wonderful for SA.

That said, you'd be hard pressed to find many accidents ADS-B would have definitively prevented. Exactly how many mid-airs are there every year? It is a very small number. Weather? Until ADS-B actually changes the weather, I struggle with claiming accident reductions as a benefit.

As I wrote a while back, most pilots would get a lot more safety bang for their bucks with more recurrent training, rather than by spending the same money on a gadget.
 
PaulS said:
Why does it have to be black or white Rob? Have you flown in an airplane with traffic reporting?
Click to expand...

Yup. C177RG, PA32-300, and I have ADS-B in on my RV. Waiting on shop schedule to install the 330ES for out.

And with all that.., not a fan.
 
Reading the link now....first thing a medical reviewer is going to look at when perusing research is its source in order to possibly detect bias. This says Regulus Group did the research then goes on to say how they contract with the FAA etc...does RG benefit from an infusion of cash into the ADS-B system full stop.
 
PeterNSteinmetz said:
The effect size here is large (surprisingly so actually) and they have estimated fairly large number of operations, so a significant effect would not be surprising.

Could be other methodological issues? It seems like a very large effect.
Click to expand...

No, it isn't. Maybe you didn't see my edit.

Let's say there are an average of 10 mid-airs per year. Let's also say that 5% of pilots are named Charlie. Let's further say that during a particular year, zero pilots named Charlie were involved in mid-air collisions. Would that surprise you? Would you think that being named Charlie helped you prevent mid-air collisions?
 
Ravioli said:
Yup. C177RG, PA32-300, and I have ADS-B in on my RV. Waiting on shop schedule to install the 330ES for out.

And with all that.., not a fan.
Click to expand...

Having more options is just too taxing ,isn’t it ? Must be hard to even tolerate your engine up there - damn all these distractions ...
 
We have to remember too that if the accident rate is for example, halved - half of nothing is still...nothing.
 
Lies, Dam* Lies and Statistics. Mark Twain.

Correlation does not verify Causation. Me (Engineer, not a Mathematician)

Cheers
 
Warmi said:
Having more options is just too taxing ,isn’t it ? Must be hard to even tolerate your engine up there - damn all these distractions ...
Click to expand...

I know I'm a bit dense, but what the **** are you asking?

Do you have some belief that ADSB is going to keep your prop spinning, or your eyes outside to see and avoid, or do anything of ANY potential value?

Other than for you to sound cool saying "Got 'em on the fish finder." Unfortunately it's going to lead to more fish getting caught, by somebody's prop.
 
It is pretty simple - it is just another option to help you out there , a poor man’s version of TCAS - nothing will guarantee you absolutely safety but why not take any additional help you can get ?
 
Ravioli said:
Let me help you ADS-B clowns out:

1) Something didn't happen today - Praise the ADS-B system

2) Something happened less frequently in our study period than our made up prior history period - Praise the ADS-B system

3) "I've got ADS-B and I don't like looking out the window" You're families will not praise the ADS-B system, unless you are heavily insured.
Click to expand...
You missed a big one.

4) I’ve got adsb and now I know about all the times I almost, but didn’t, hit someone that before adsb I wouldn’t have even worried about.

I’d like to see a study that looks at whether having this info causes people to change their tracks more erratically increasing the possibility of an accident.

The odds of two planes on a straight and level course actually being in the same spot at the same altitude at the same moment really are very small.

PS> I use adsb in.
 
Space exploration is bad due to the increase in suicides.
chart.jpeg
Reducing consumption of Margarine reduces the divorce rate
chart(1).jpeg
Preventing marrige in Kentucky reduces drowning deaths.
chart(2).jpeg
Buy American! Japanese cars cause suicides!
chart(3).jpeg
 
Was out in the Long Beach practice area a few days ago doing radio calls every 3-5 minutes doing maneuvers and all of a sudden I get a traffic alert same altitude, he was heading right for me. I called out my location and altitude and started looking, couldn't see him and no responses. I finally spotted him and dove down, he never budged. Without ADSB I may have become a statistic.
 
Jamie Kirk said:
Without ADSB I may have become a statistic.
Click to expand...

Maybe, but odds are a million to one that you'd have blindly missed, or that one would have seen the other and would have engineered a miss.
 
Salty said:
You missed a big one.

4) I’ve got adsb and now I know about all the times I almost, but didn’t, hit someone that before adsb I wouldn’t have even worried about.

I’d like to see a study that looks at whether having this info causes people to change their tracks more erratically increasing the possibility of an accident.

The odds of two planes on a straight and level course actually being in the same spot at the same altitude at the same moment really are very small.

PS> I use adsb in.
Click to expand...

I disagree with this Salty, the big sky theory is a very poor collision avoidance guarantee. Just about every flight around here I need to adjust or be vigilant about another aircraft to avoid intersecting paths. The last one was last week, I was at 2,500 feet, he was 3,000 feet. Pointed out to us by ATC, we were on flight following, he was not. We got visual, he turned and flew right over us. Not sure but I don't think he had a clue we were there. We have ads b in/out with traffic and we didn't even look at that screen instead of outside, it was really hard though. :eek:
 
While I think it's too early to cite statistics showing it's responsible for an "impressive" reduction in accidents, common sense says it has the potential to significantly reduce midairs and weather related accidents when the majority of traffic is equipped with In and Out.

I'm a believer.
 
PaulS said:
I disagree with this Salty, the big sky theory is a very poor collision avoidance guarantee. Just about every flight around here I need to adjust or be vigilant about another aircraft to avoid intersecting paths. The last one was last week, I was at 2,500 feet, he was 3,000 feet. Pointed out to us by ATC, we were on flight following, he was not. We got visual, he turned and flew right over us. Not sure but I don't think he had a clue we were there. We have ads b in/out with traffic and we didn't even look at that screen instead of outside, it was really hard though. :eek:
Click to expand...
500 feet, plus likely some amount offset to the side even if it seemed “right overhead”? You pass next to cars 10 feet away at a 100+mph speed delta all the time.
 
dmspilot said:
No, it isn't. Maybe you didn't see my edit.

Let's say there are an average of 10 mid-airs per year. Let's also say that 5% of pilots are named Charlie. Let's further say that during a particular year, zero pilots named Charlie were involved in mid-air collisions. Would that surprise you? Would you think that being named Charlie helped you prevent mid-air collisions?
Click to expand...

I think the binomial test should account for this. It is basically a test for a difference in ratios. In this case the fraction of equipped aircraft which have accidents vs the fraction of unequipped aircraft which have accidents.

They did not give the total operations which they are using for the denominator, though I gather it is in a general report.

I agree and would be surprised by the result in your example. However, the numbers they are using are different than that. 5% of the total number of GA operations under consideration is still a fairly large number. What would be needed to double check them is the values of the denominators in the ratios from the report they reference.
 
Let'sgoflying! said:
Reading the link now....first thing a medical reviewer is going to look at when perusing research is its source in order to possibly detect bias. This says Regulus Group did the research then goes on to say how they contract with the FAA etc...does RG benefit from an infusion of cash into the ADS-B system full stop.
Click to expand...

I agree there are a lot of possible biases that could cause the FAA or their contractors to want to over-inflate the benefits of ADS-B.

This type of argument though is a subtle form of attack on the source or speaker. Technically such bias does not indicate whether the argument is correct or not because even fools can occasionally speak the truth.

That is one reason I don’t entirely agree with the current policy of requiring disclosures of conflicts in medical journals. I think we should look for better ways of improving reviews and the accuracy and replicability of papers rather than trying to ferret out biases.

Disclosure of potential conflicts is required now though. OTOH, I know of no reviewers who make that a first concern. I usually find it more productive to focus on the methods and results and see if they are correct.
 
Salty said:
500 feet, plus likely some amount offset to the side even if it seemed “right overhead”? You pass next to cars 10 feet away at a 100+mph speed delta all the time.
Click to expand...

That was the latest Salty, we were at 3,000 feet, going 165 knots before ATC pointed them out. At least once every other flight, sometimes more I get turned for traffic. With big sky theory that should never happen. The issue is we are all flying to and from a limited number of fixed points and it's bound to happen. I like the extra tools.
 
Capt. Geoffrey Thorpe said:
Reducing consumption of Margarine reduces the divorce rate
Click to expand...

It is a good point that correlation does not imply causality. But that does not mean that all such correlational studies should simply be discarded.

Rather there are other indicators of causality that need to be present to suggest causality.

One such is plausible mechanism. Is it plausible that having information about the location of other aircraft and weather in the cockpit helps pilots to avoid collisions and fatalities? Seems at least plausible to me.

Another is the lack of other potential causative factors. Are there other possible things happening in GA in these 4 years which would have caused this near halving of accidents in the planes equipped with ADS-B? I will leave it to others more knowledgeable than me to suggest those.
 
PeterNSteinmetz said:
I think the binomial test should account for this. It is basically a test for a difference in ratios. In this case the fraction of equipped aircraft which have accidents vs the fraction of unequipped aircraft which have accidents.
Click to expand...

I didn't read the report cover to cover. But I see now you're right, they did check for statistical significance. And you're wrong, there was none as far as mid-airs are concerned.

Screen Shot 2019-04-19 at 9.16.02 PM.png
 
You must log in or register to reply here.
Back
Top