Have the written tests become obsolete?

O

old cfi

Pre-takeoff checklist
Gone West
Joined
Jun 18, 2015
Messages
373
Location
SW Missouri
Display Name
Display name:
Old CFI
Since the FAA has decided DPE''s need to hit on everything in the ACS, like looooooong orals covering much of the info in the written, do you think testing an applicant face-to-face during the checkride is much more effective than taking a written test where many of the questions are out there in one form or another? Just curious as to what others think.
 
At the very least they serve as a good screening tool, preventing the totally unprepared from wasting a DPE's time.
 
Have to disagree with you, midwestpa24. His CFI is wasting the DPE's time for signing him off for the ride if he is not fully prepared!
 
If anything, the orals have become obsolete. We have the technology to create comprehensive written exams that adequately test an applicant's understanding of the material. The problem is that the FAA has never been able to create one. I've taken exams that were far better made, and far more in depth, where every question was crafted not to trick you, but to truly test the depth of your understanding. I've taken adaptive exams that locked onto your weaknesses and drilled into them. Yet, the FAA continues to cling to outdated methodologies, and for no logical reason. Hell, the exam I had to take for Angel Flight was more effective at scenario based testing.

All an oral exam does is test your ability to articulate your understanding, which some people aren't good at. I'd like to see the ground portion of the exam concentrate on reviewing your proposed flight plan, and why you made the planning decisions you made. Then take the time you saved by not having that ridiculous oral and use it for the flight portion, which is the only thing that really cannot be tested any other way.

Of course, nothing like this will ever happen. Perhaps instead of privatizing ATC, we should be privatizing pilot training AND testing, and get the FAA out of it.
 
old cfi said:
Since the FAA has decided DPE''s need to hit on everything in the ACS, like looooooong orals covering much of the info in the written, do you think testing an applicant face-to-face during the checkride is much more effective than taking a written test where many of the questions are out there in one form or another? Just curious as to what others think.
Click to expand...

Since that's NOT what the FAA intended, and says so in their monthly seminars about the ACS... the premise of the question is busted.

Highly recommend you attend one online. Free, and informative. I attended in a pair of shorts with my feet up on my deck railing on a wifi iPad last spring. Tech worked great. They're still offering them, AFAIK.

Rykymus said:
All an oral exam does is test your ability to articulate your understanding, which some people aren't good at. I'd like to see the ground portion of the exam concentrate on reviewing your proposed flight plan, and why you made the planning decisions you made. Then take the time you saved by not having that ridiculous oral and use it for the flight portion, which is the only thing that really cannot be tested any other way.

Of course, nothing like this will ever happen. Perhaps instead of privatizing ATC, we should be privatizing pilot training AND testing, and get the FAA out of it.
Click to expand...

If scenario based orals aren't what's happening in your local area, something's wrong at your FSDO and with your DPEs. The direct word from the aforementioned seminars from the makers of the ACS itself, say so.

I'll happily report that at least two local DPEs here are doing It right. Build the right scenarios and covering all the items in the ACS isn't all that hard, it's just a list of stuff folks do and know on every flight. And even then, not all of what they know and do... it's a MINIMUM standard, as Doc Bruce would say.

I was pretty impressed with my Commercial and Initial CFI orals and have heard the Private oral plan at least one area DPE shoots for. His scenarios are impressive because he WORKED on them. He covers the entire ACS without the applicant feeling like he's working through a checklist of items by making those items things you'd need to know to accomplish the given scenario and flight. He also bases those scenarios off of a starting flight plan that is the applicant's.

A simple example. Applicant is told to plan a flight. They discuss that plan. Examiner then says, "Weather along the route of flight has changed. Here's a sheet with some of the METARs and TAFs. Tell me what you think and what you'd do."

Scenarios done right with open-needed questions and a savvy examiner cover the ACS items just fine. Often a pile of them in a single scenario change. It is on the examiner to do it the prescribed way, however, and not just read down a list of items.

Glad we have some here who get that. Good folks. Work hard. Make some serious bucks, too, but I'm okay with that. They're not just sitting around between exams eating bon-bonds on the couch. I could see where if some are, they wouldn't be up to speed yet on scenario based examinations, and ACS has been out plenty long that they should have done the prep work for that by now.
 
c0d7369881dea069d767f669bdf3fef8.jpg


Bon bons..... good....
 
Funny, a lot of folks who poo poo the FAA knowledge Tests (they haven't been written tests for at least 25 years) haven't taken one recently. The FAA has been hard at work for years removing obsolete questions from the test bank and adding newer relevant ones that correspond to the appropriate ACS. The knowledge test is one tool to ensure the applicant has broad knowledge of the topic area; you can't test for everything in the oral portion of the check ride.
 
Brad Z said:
Funny, a lot of folks who poo poo the FAA knowledge Tests (they haven't been written tests for at least 25 years) haven't taken one recently. The FAA has been hard at work for years removing obsolete questions from the test bank and adding newer relevant ones that correspond to the appropriate ACS. The knowledge test is one tool to ensure the applicant has broad knowledge of the topic area; you can't test for everything in the oral portion of the check ride.
Click to expand...
Adding newer, somewhat less irrelevant questions, related to the ACS donkey? Concur, you can't test for the entire ocean of minutia that exists, or account for the FAA's inability to discern the essential from the trivial, while giving equal weight to both. The tests were/are, at best, an outline of the knowledge required to satisfy the FAA, which is not/not the same thing as that required to be a competent pilot.

PS
Did U really associate "FAA" and "hard at work" ???? :)
 
Sundancer said:
Adding newer, somewhat less irrelevant questions, related to the ACS donkey? Concur, you can't test for the entire ocean of minutia that exists, or account for the FAA's inability to discern the essential from the trivial, while giving equal weight to both. The tests were/are, at best, an outline of the knowledge required to satisfy the FAA, which is not/not the same thing as that required to be a competent pilot.

PS
Did U really associate "FAA" and "hard at work" ???? :)
Click to expand...
Shockingly, yes I did. The people who write those questions are my colleagues.
 
Any shoulder injuries among them, from patting themselves on the back? Any sleepless nights from using "Safety" as cover for sucking the soul out of aviation? Or the Larry Lightbulb ADS-B design? Or the so consistent application of certificate actions?

Just my opinion, I could be wrong. . . You may have discerned I favor the Martha Lunken view of the FAA. . .
 
I think the whole written and DPE/pts/acs system sucks. Written just becomes drill and kill type rote test question knowledge and the DPE's around here seemingly have their own exams. Was reading the flight schools "practical test prep gouge" for various rides and DPE's and it was about 5 pages of don't wear shorts, don't wear aviators, don't dress too nice though, he'll ask this obscure question and this is the answer he's looking for. He'll combine your short field and soft field landings into one. He'll give you grief over using simulator time for IR, don't use an ipad unless its this DPE then preferably use it.

Reading practical test write ups from various people across the country the variation on rides seems extreme some CFI orals lasting 8+ hours others done before lunch only hitting the minimum in the PTS, some bust multiple maneuvers and get to "redo" them or get a pass on it. Other fail instantly. The guy i'm probably gonna have to use for my commercial I know he's gonna tell me first thing "I wanna see a power off 180 and I want it to be soft field"(not in pts btw) and CFI said the whole ride pretty much hinges on that. If you get the 180 but its not soft you continue but unless your bob hoover for the rest of flight your gonna be coming back for round two.

Some guys IFR ride was a big deal, mine was go up do a quick hold, one "unusual attitude" by which he basically just put me in a climb and told me to recover at IAP altitude, then 3 approaches. My partial panel was covering up the speedo "your speedos not working its cold out what happened" pitot could have iced up so pitot heat on. He said "yep that fixed it" and took the cover off. All 15 seconds cruising at 90 kts down an ILS oooohhh so hard. Ride was done in 45 min or so.

Written doesn't test is still bad at least the IR when i took it fall 2016 the test prep for CP doest lend me any faith that its any less useless than the ppl and ir were.
TL DR written test is kinda dumb, DPE variation sucks.
 
Last edited:
Brad Z said:
Funny, a lot of folks who poo poo the FAA knowledge Tests (they haven't been written tests for at least 25 years) haven't taken one recently. The FAA has been hard at work for years removing obsolete questions from the test bank and adding newer relevant ones that correspond to the appropriate ACS. The knowledge test is one tool to ensure the applicant has broad knowledge of the topic area; you can't test for everything in the oral portion of the check ride.
Click to expand...

Not buying it. I assist students with their knowledge tests on a regular basis. The test questions are as bad as ever. Sometimes they even make the test worse (for example, messing up the chart scales in 2013; at least now they have a note on them saying they're not to scale).
 
Brad Z said:
Funny, a lot of folks who poo poo the FAA knowledge Tests (they haven't been written tests for at least 25 years) haven't taken one recently. The FAA has been hard at work for years removing obsolete questions from the test bank and adding newer relevant ones that correspond to the appropriate ACS. The knowledge test is one tool to ensure the applicant has broad knowledge of the topic area; you can't test for everything in the oral portion of the check ride.
Click to expand...

I took mine(IR) a couple of weeks ago, and yes it still has tons of obsolete/ambiguous/useless crap. And looking at the CP questions, there's plenty of that to look forward to (how far are you from x if y moves by z in 38 seconds).
If they were hard at work, they could fix the question bank in a week.
Big problem in the knowledge test method is, that many questions are missing the _actual_ correct answer which is what you do in an airplane. For example, for the knob twist questions, correct answer is "you twist it one way and if it's the wrong way, you twist it the other way".
 
mtuomi said:
Big problem in the knowledge test method is, that many questions are missing the _actual_ correct answer which is what you do in an airplane. For example, for the knob twist questions, correct answer is "you twist it one way and if it's the wrong way, you twist it the other way".
Click to expand...
Since this one comes up frequently lately... ;)

If you were writing an exam question on remote indicating compasses that had to be both "operational" (as opposed to chasing electrons or building/maintaining the system), and you wanted it to be as generic as possible so as not to be specific to one brand/model, how would you write it, and what would be your distractors?
 
mtuomi said:
I took mine(IR) a couple of weeks ago, and yes it still has tons of obsolete/ambiguous/useless crap. And looking at the CP questions, there's plenty of that to look forward to (how far are you from x if y moves by z in 38 seconds).
If they were hard at work, they could fix the question bank in a week.
Big problem in the knowledge test method is, that many questions are missing the _actual_ correct answer which is what you do in an airplane. For example, for the knob twist questions, correct answer is "you twist it one way and if it's the wrong way, you twist it the other way".
Click to expand...
From the AFS-630 FAQ:

What recourse do I have if I believe there was an invalid question on my test?

If you feel that your test contained an invalid question, you may request that your test be hand scored. If an applicant wishes to have a test hand-scored, he or she must submit a request, in the form of a signed letter, to the Airman Testing Standards Branch, AFS-630. Along with the request, the applicant must submit a legible photocopy of proof of identification, including an official photograph of the applicant and his or her signature. The mailing address is:

U.S. Department of Transportation
Federal Aviation Administration
Airman Testing Standards Branch, AFS-630 P.O. Box 25082
Oklahoma City, OK 73125
 
old cfi said:
Since the FAA has decided DPE''s need to hit on everything in the ACS, like looooooong orals covering much of the info in the written, do you think testing an applicant face-to-face during the checkride is much more effective than taking a written test where many of the questions are out there in one form or another? Just curious as to what others think.
Click to expand...

I don't think the face-to-face is more effective, although it could be depending on the DPE. I think they complement each other. Some folks do better at face-to-face while others do better on a written test. Both types of people may be equally qualified. Does the DPE still look at the written test results as he/she formulates their interview?
 
SixPapaCharlie said:
Which way do I turn the knob on a remote indicating compass? If I get this wrong during a flight, ill not know what to do.
Click to expand...
Somewhere over southern Georgia on this day in 2017, I was given a new altimeter setting. I spun the knob the wrong way, had a flashback, and then froze in my seat, sweating profusely.

I didnt know what to do.
 
Brad Z said:
From the AFS-630 FAQ:

What recourse do I have if I believe there was an invalid question on my test?

If you feel that your test contained an invalid question, you may request that your test be hand scored. If an applicant wishes to have a test hand-scored, he or she must submit a request, in the form of a signed letter, to the Airman Testing Standards Branch, AFS-630. Along with the request, the applicant must submit a legible photocopy of proof of identification, including an official photograph of the applicant and his or her signature. The mailing address is:

U.S. Department of Transportation
Federal Aviation Administration
Airman Testing Standards Branch, AFS-630 P.O. Box 25082
Oklahoma City, OK 73125
Click to expand...

Define "invalid question."

The FAA is going to say "Is this a piece of equipment in some airplanes? Yes. Is the answer to our question correct? Yes. Valid question."

I hate the stupid HSI questions. You know how many I've seen? Zero. There is not a single HSI in any of the 6 aircraft I've flown. I certainly don't have one in the aircraft I own. But I'm responsible for knowing how it works and what it says. Garbage, imho.
 
Rykymus said:
If anything, the orals have become obsolete. We have the technology to create comprehensive written exams that adequately test an applicant's understanding of the material. The problem is that the FAA has never been able to create one. I've taken exams that were far better made, and far more in depth, where every question was crafted not to trick you, but to truly test the depth of your understanding. I've taken adaptive exams that locked onto your weaknesses and drilled into them. Yet, the FAA continues to cling to outdated methodologies, and for no logical reason. Hell, the exam I had to take for Angel Flight was more effective at scenario based testing.

All an oral exam does is test your ability to articulate your understanding, which some people aren't good at. I'd like to see the ground portion of the exam concentrate on reviewing your proposed flight plan, and why you made the planning decisions you made. Then take the time you saved by not having that ridiculous oral and use it for the flight portion, which is the only thing that really cannot be tested any other way.

Of course, nothing like this will ever happen. Perhaps instead of privatizing ATC, we should be privatizing pilot training AND testing, and get the FAA out of it.
Click to expand...
Don't like the idea. I think an oral is the best test of the three (written, oral, practical).
 
ircphoenix said:
Define "invalid question."

The FAA is going to say "Is this a piece of equipment in some airplanes? Yes. Is the answer to our question correct? Yes. Valid question."
Click to expand...
Agree.

Low quality question =/= Invalid question
 
Frankly, they've always been obsolete. They're a speed bump to certification. Such is the case with most such exams, and the FAA ones are particularly bad.
 
ircphoenix said:
I hate the stupid HSI questions. You know how many I've seen? Zero. There is not a single HSI in any of the 6 aircraft I've flown. I certainly don't have one in the aircraft I own. But I'm responsible for knowing how it works and what it says. Garbage, imho.
Click to expand...
That's why it's on the written test, not the oral. And having it on the written test is far superior to an oral or checkride for each new piece of equipment you see in your lifetime.
 
Wouldn't the best way to see if you're an instrument pilot be to actually make a trip?

Written, oral, check ride.... they're not effective ways of determining what you'll really do. So make the check ride into a long cross country. Most check rides take half a day anyway. Go someplace. Examiner can ask questions along the way. No need for "scenario based exams"... you're IN a chain of scenarios that make sense and flow correctly. Heck, make it, say, 4 hours of cross country with a DPE for your Instrument "sign off". Once you demonstrate you can successfully make a few IFR flights, you're an instrument pilot. Period. Everything else is ********.
 
I don't think they should do away with the written but I think there should be more of an emphasis on things you'd need to know immediately in flight and less of an emphasis on regulatory minutia you could easily look up on the ground.

That and the calculation questions where they give you a couple of answers that are close enough together that the little bit of play in your cardboard e6b makes it unclear which one is right. I doubt many new pilots are going to use them after their exam is done with anyway.
 
cowman said:
That and the calculation questions where they give you a couple of answers that are close enough together that the little bit of play in your cardboard e6b makes it unclear which one is right. I doubt many new pilots are going to use them after their exam is done with anyway.
Click to expand...
^^^ This. In several cases the difference between the correct answer and one or two of the others was in the roundoff error noise.
 
Mistake Not... said:
Wouldn't the best way to see if you're an instrument pilot be to actually make a trip?

Written, oral, check ride.... they're not effective ways of determining what you'll really do. So make the check ride into a long cross country. Most check rides take half a day anyway. Go someplace. Examiner can ask questions along the way. No need for "scenario based exams"... you're IN a chain of scenarios that make sense and flow correctly. Heck, make it, say, 4 hours of cross country with a DPE for your Instrument "sign off". Once you demonstrate you can successfully make a few IFR flights, you're an instrument pilot. Period. Everything else is ********.
Click to expand...
That wouldn't be 4 hours...it'd be a couple of days.
 
If you can understand a normal CDI, you can understand an HSI. I had zero problem adapting to my HSI (written tests notwithstanding). It's kind of silly to waste time on the test with them.
 
flyingron said:
If you can understand a normal CDI, you can understand an HSI. I had zero problem adapting to my HSI (written tests notwithstanding). It's kind of silly to waste time on the test with them.
Click to expand...
Same here. I started flying with an HSI about a year after getting my PPL and was very quickly hooked on it. In an airplane without a moving map GPS, it's one of the best situational awareness aids I know of. Even though I do have a (installed) moving map now, I wouldn't want to go back to an old-style CDI. The HSI conveys the same information as a DG and a CDI in an easily understood format and much more compactly.

Still, if my first exposure to an HSI had been while studying for the written, I might have felt differently...
 
At $2.50 a question, the written test is valuable to somebody.
 
I took the CFI-II test recently. It still needs work, but the really insane questions were not there. I took the CFI-A written last summer, there no 'barbell W&B' problems, no absurd interpolation problems. I think the FAA is moving in the correct direction with respect to the written tests.
 
cowman said:
I don't think they should do away with the written but I think there should be more of an emphasis on things you'd need to know immediately in flight and less of an emphasis on regulatory minutia you could easily look up on the ground.
Click to expand...

The stuff you need to memorize and understand in flight is what's most critical. I recall thinking that a lot of the questions fell into the "well, if I need to know that, I'll grab the AIM and look it up" category.

Although I don't get why so many people here think the HSI questions are bunk. That's a completely legitimate subject area. Most high-performance IFR-capable aircraft are likely to have one, and 100% of glass aircraft have them. Even if your plane doesn't have one, the questions do a good job probing whether you understand how a CDI actually works. We're all creating HSIs in our head when we learn; the instrument is just doing the work for you and actually making life easier (a LOT easier if you're flying an approach with a big crosswind, incidentally). If you can't wrap your mind around overlaying a CDI on your DG, you probably aren't quite ready to fly in the system. That also fits into the "in-flight critical" category as well. Messing up the HSI configuration for a back-course localizer approach could very well send you into a mountain.
 
Those of us with PFD/MFD/HSI and dual ADHRS (redundant GPS WAAS) resent an analog six-pack, VORs, DME, ADFs and CDIs. We would like no test questions on the old tech.
 
JimNtexas said:
I took the CFI-II test recently. It still needs work, but the really insane questions were not there. I took the CFI-A written last summer, there no 'barbell W&B' problems, no absurd interpolation problems. I think the FAA is moving in the correct direction with respect to the written tests.
Click to expand...

Ugh. I hate the barbell crap. I don't do W&B that way, I just rework the the changes. I get the barbell, it just seems silly -- I want a new takeoff and landing weight and zero fuel weight on the graph.

The gadgetry of course makes this insanely simple, but even re-doing it with a pencil doesn't take much time and you end up with a correct sheet of info and one you can throw out.
 
denverpilot said:
Since that's NOT what the FAA intended, and says so in their monthly seminars about the ACS... the premise of the question is busted.

Highly recommend you attend one online. Free, and informative. I attended in a pair of shorts with my feet up on my deck railing on a wifi iPad last spring. Tech worked great. They're still offering them, AFAIK.



If scenario based orals aren't what's happening in your local area, something's wrong at your FSDO and with your DPEs. The direct word from the aforementioned seminars from the makers of the ACS itself, say so.

I'll happily report that at least two local DPEs here are doing It right. Build the right scenarios and covering all the items in the ACS isn't all that hard, it's just a list of stuff folks do and know on every flight. And even then, not all of what they know and do... it's a MINIMUM standard, as Doc Bruce would say.

I was pretty impressed with my Commercial and Initial CFI orals and have heard the Private oral plan at least one area DPE shoots for. His scenarios are impressive because he WORKED on them. He covers the entire ACS without the applicant feeling like he's working through a checklist of items by making those items things you'd need to know to accomplish the given scenario and flight. He also bases those scenarios off of a starting flight plan that is the applicant's.

A simple example. Applicant is told to plan a flight. They discuss that plan. Examiner then says, "Weather along the route of flight has changed. Here's a sheet with some of the METARs and TAFs. Tell me what you think and what you'd do."

Scenarios done right with open-needed questions and a savvy examiner cover the ACS items just fine. Often a pile of them in a single scenario change. It is on the examiner to do it the prescribed way, however, and not just read down a list of items.

Glad we have some here who get that. Good folks. Work hard. Make some serious bucks, too, but I'm okay with that. They're not just sitting around between exams eating bon-bonds on the couch. I could see where if some are, they wouldn't be up to speed yet on scenario based examinations, and ACS has been out plenty long that they should have done the prep work for that by now.
Click to expand...

Is that DPE you're talking about named Craig?
 
Lachlan said:
Is that DPE you're talking about named Craig?
Click to expand...

No, I (maybe unfortunately) haven't had a ride with Craig (yet). I've met him a number of times and he's a great guy. He's been incredibly busy lately, got a new wife and (more) kiddos, instructing, DPE stuff, going back to school at Denver Seminary, and flying charter.

See his FB posts all the time but timing just hasn't worked out to call him for a checkride. First met him in a parking lot of a nearby restaurant a few years ago after we struck up a conversation about his license plates. Heh. From the looks of yesterday's posts he was probably in the pattern with me. Ha. But different airplane. ;-)
 
He flies the heck out of that Falcon! He's a really great guy. The kind of guy who can teach you a lot in seconds, if you're paying attention, and you don't even realize he was handing you vast amounts of knowledge. Teachers like that are rare. It's a gift, and he's got it. Wonderful human being, amazing instructor. Very very good as a DPE. And his license plate is pretty good- they do tell his story in 7 characters!
 
Last edited:
You must log in or register to reply here.
Back
Top