Commercial written (mild rant)

C

CerroTorre

Pre-takeoff checklist
Joined
Nov 1, 2020
Messages
136
Display Name
Display name:
CerroTorre
I’ll be taking the Commercial written in the next few weeks and am most of the way through a test prep course at the moment. I’m trending fine, have a pretty good grasp of most of the material, have a good track record on FAA exams (short though it may be), etc. That said, the commercial test, for some reason, seems to be significantly more prone to problematic wording, “gotcha” questions, and just plain wrong answers than either the IR or PPL (where it was still a problem). My annoyance with whoever runs the testing department at the FAA is currently at an all time high. Hours of wading through the result of either incompetence or basic laziness at “work” doesn’t really help my mood :D … It’s just so bad. I picture the test writing division like the Dunder-Mifflin office of the FAA…comically bad. With some manager more interested in running around hard-core parkour-ing off water coolers and trash cans…

But whatever. It is what it is. Deal with it, just like everyone else I guess.=)

That said, my stress level (never great for exams in general) is a bit higher than it should be.

Has the FAA ever responded to criticism of the tests in a formal way? Has anyone ever had a conversation with anyone there about how poorly the tests are written? Is there an actual philosophy that tries to support the “approach” (if we can call it that)? Anyone actually know someone in the test writing office?

There are so many errors and questionably written questions, it almost seems deliberate. Which begs a lot of … um … questions.

I think most of us just roll our eyes and move on as soon as possible after taking each exam. So it ends up getting very little pressure to change. Could it not be a lot better? Maybe test actual, real world knowledge? Is it not worth it? It’s only a “gate-keeping” exam, not a real examination of knowledge? Am I barking up the wrong tree? Too many other more important things to change and improve first?

Ok, rant over. Just taking a study break.

sorry, edits done now. =)
 
Last edited:
eman1200 said:
you haven't even taken the test, how do you know there are "so many errors and questionably written questions" ?
Click to expand...
Current study methodologies have it pretty well established. Plus confirmed from other’s experience. And my own on previous exams.

Not trying to start an argument.
 
eman1200 said:
you haven't even taken the test, how do you know there are "so many errors and questionably written questions" ?
Click to expand...
The only alternative is to blame yourself for your own test taking stress level on a test you prepare for by reading most the questions and answers and learn tricks to dope out the rest. Kind of makes one wonder about what's wrong with all those people who score in the 90s.

But to answer your question, @CerroTorre, yeah, people have been telling the FAA their multiple guess knowledge tests suck for decades. Believe it or not, they've actuallly gotten better.
 
As sad as it may be that they were once worse, I’ll take the wins where we can get them. =)

Still, the OCD side in me would like to just be a fly on the wall in that office sometime. “What … what would you say you do here?”

Oh well. I’ll pass regardless.
 
If you think FAA test are bad, go look at the test and requirements for test in EASA, or other countries.
 
I suspect they do that a bit on purpose. Written tests are just a stepping stone before you get to the Oral test preparation.

If they just ask a basic question and you answer, then you know the answer.

If they ask a question with a questionable answer you are more likely to research the subject and learn why some answers are more correct than others. As a result you will now likely understand the subject better than if you just answered a simple question.

Brian
CFIIG/ASEL
 
In my entire career.......retired now.....nobody EVER asked to see written results. I burned a million gallons in these airplanes.

Lear Jet
Cessna 500
Falcon 50
King Air 350
 
CerroTorre said:
I’ll be taking the Commercial written in the next few weeks and am most of the way through a test prep course at the moment. I’m trending fine, have a pretty good grasp of most of the material, have a good track record on FAA exams (short though it may be), etc. That said, the commercial test, for some reason, seems to be significantly more prone to problematic wording, “gotcha” questions, and just plain wrong answers than either the IR or PPL (where it was still a problem). My annoyance with whoever runs the testing department at the FAA is currently at an all time high. Hours of wading through the result of either incompetence or basic laziness at “work” doesn’t really help my mood :D … It’s just so bad. I picture the test writing division like the Dunder-Mifflin office of the FAA…comically bad. With some manager more interested in running around hard-core parkour-ing off water coolers and trash cans…

But whatever. It is what it is. Deal with it, just like everyone else I guess.=)

That said, my stress level (never great for exams in general) is a bit higher than it should be.

Has the FAA ever responded to criticism of the tests in a formal way? Has anyone ever had a conversation with anyone there about how poorly the tests are written? Is there an actual philosophy that tries to support the “approach” (if we can call it that)? Anyone actually know someone in the test writing office?

There are so many errors and questionably written questions, it almost seems deliberate. Which begs a lot of … um … questions.

I think most of us just roll our eyes and move on as soon as possible after taking each exam. So it ends up getting very little pressure to change. Could it not be a lot better? Maybe test actual, real world knowledge? Is it not worth it? It’s only a “gate-keeping” exam, not a real examination of knowledge? Am I barking up the wrong tree? Too many other more important things to change and improve first?

Ok, rant over. Just taking a study break.

sorry, edits done now. =)
Click to expand...
Sounds like whatever study guide you’re using sucks. The FAA doesn’t release the question bank (and hasn’t for more than 20 years), and test prep companies have no reliable way of determining what questions are added or what questions have been removed. Some test prep companies will reward test takers for sharing test questions they encountered, but because you cannot walk out of a testing center with notes, the test companies are relying on test taker’s memory. Many test prep companies are lazy and will leave old questions in because they have no incentive to remove them.

Plenty of people have complained about studying for tests, but no one who has properly prepared for the test walked out of a test feeling like the FAA asked questions that were outside of the appropriate ACS knowledge areas for the rating sought.
 
brcase said:
I suspect they do that a bit on purpose. Written tests are just a stepping stone before you get to the Oral test preparation.
Click to expand...

From a pedagogical position, that’s the dumbest thing a test writer can do and quite possibly invalidates the test, which is why most professional
knowledge validation sources will always be piloting new questions with tremendous rigor. For example,
It’s fairly normal for a 60 question test to have 65 questions with five ungraded pilot questions being validated. The FAA states their written exams may include questions being validated, but if they stay with 60 AND pilot question get thrown out, then the number of questions that must be answered correctly changes and that brings about its own set of problems.
 
Brad Z said:
The FAA doesn’t release the question bank (and hasn’t for more than 20 years)
Click to expand...
Not quite. The last time they released the test bank was either 2014 (I have copies) or 2015. I know for sure that they did not release the test bank in 2016.
 
I just took the CPL written test yesterday. I thought there were no gotcha questions and most of the right answers were very easy to spot, some options were even comedic.
I thought it was easier then the test prep. Passed easily.
 
I took it a few weeks ago. Quite a bit on the test that I hadn’t seen in prep (Gleim), and honestly I
thought I might have failed. Got an 88, which was about my average on all of the pre-tests I took.
 
I told the FAA basically this for IR and Commercial and they agreed. They said further improvements are in the works.
 
Brad Z said:
Plenty of people have complained about studying for tests, but no one who has properly prepared for the test walked out of a test feeling like the FAA asked questions that were outside of the appropriate ACS knowledge areas for the rating sought.
Click to expand...

Straw man argument. OP complained about poorly worded, tricky, and incorrect questions, not ones testing irrelevant knowledge areas.
 
dmspilot said:
Straw man argument. OP complained about poorly worded, tricky, and incorrect questions, not ones testing irrelevant knowledge areas.
Click to expand...

Yes, for a test they have not yet taken.

In other news, being an astronaut sucks, the Gs are totally too high and nobody tells you about the gastrointestinal effects of a rocket launch. I'm not trying to start an argument, just take my tablets of wisdom from the mountain and applaud or console me.

:rolleyes:
 
schmookeeg said:
Yes, for a test they have not yet taken.

In other news, being an astronaut sucks, the Gs are totally too high and nobody tells you about the gastrointestinal effects of a rocket launch. I'm not trying to start an argument, just take my tablets of wisdom from the mountain and applaud or console me.
Click to expand...

But there are poorly worded, tricky, and incorrect questions. I have taken enough writtens to know. No idea what the rest of your blathering is supposed to mean. :rolleyes:
 
dmspilot said:
But there are poorly worded, tricky, and incorrect questions. I have taken enough writtens to know. No idea what the rest of your blathering is supposed to mean. :rolleyes:
Click to expand...
When is the last knowledge test you’ve taken? Which test was it?
 
MacFlier said:
I just took the CPL written test yesterday. I thought there were no gotcha questions and most of the right answers were very easy to spot, some options were even comedic.
I thought it was easier then the test prep. Passed easily.
Click to expand...
Thanks MacFlier, This was the point I was making to the OP.
 
Brad Z said:
When is the last knowledge test you’ve taken? Which test was it?
Click to expand...

Was there some kind of audit recently in which all low-quality questions were deleted? By what standard would a question be judged low-quality? Considering all the mistakes in the FAA source material I sure wouldn't expect them to be able to competently accomplish such a task.
 
We’ll see when I take the test. I remain open minded… =)

So far my test prep has proven pretty darn accurate to what I’ve seen on the tests themselves. Ymmv of course, but I think the quality has been well proven.

As to the “you should learn the material then you wouldn’t have a problem” posters… Well, yeah. Of course. But you’re actually making my point for me. If the FAA made a decent exam, “knowing the material” would translate to passing with flying colors. The problem is that the current approach actually promotes studying to simply pass the test. The applicant is effectively forced to memorize what gets graded correct … even when it’s wrong. Some questions, if answered with real world knowledge (or logic) would get graded as incorrect. So … we grind it out and memorize.

And meanwhile, if we’re conscientious and responsible, we learn the material itself. So we can pass the oral/practical AND hopefully be competent, well rounded pilots. And the written gets reduced to just a pointless speed bump along the way. One that adds needless work and stress for no real, identifiable benefit.

My point is that pointless speed bumps are stupid. And the FAA should do better.

Cheers everybody. it’s not really that big a deal in the end. Just annoying.
 
It has been my opinion through many areas of study that there are two areas when trying to achieve a goal that is based upon a test.
1) Know the material being covered
2) Study for the test

Too many people seem to think the above are mutually exclusive, when they are not.

There are many people who know how to drive a car safely on the road (yes, varies by state, but still), but how many of those same people could pass a pop quiz of the written DMV test they took when they got their license? Or even more to the point, how many actually know the laws that govern vehicle operations? Yes, I'm sure there are a few POA members who have memorized all of the laws, subsections, case law, etc regarding their state's driving laws (and surrounding states they will drive through as well) but most people have not. So, they would probably want to study up for a written test (or heaven help us, a practical driving test).

I have seen cases where people obviously have the particular piece of paper, but you can tell they have no knowledge of the material.
I have also seen people who knew the material, but didn't concentrate on studying the fine details that will be tested and did poorly on the exam.

So my advice has always been to use test prep where possible, do learn the answers for specific gotcha questions, pass the test. But, before you do, make sure you know the material as that is what is going to (in the case of flying) allow you to pass the oral exams but more importantly it might save your life or your career.
 
Brad Z said:
Sounds like whatever study guide you’re using sucks. The FAA doesn’t release the question bank (and hasn’t for more than 20 years), and test prep companies have no reliable way of determining what questions are added or what questions have been removed. Some test prep companies will reward test takers for sharing test questions they encountered, but because you cannot walk out of a testing center with notes, the test companies are relying on test taker’s memory.
Click to expand...
Pretty sure there was a time before test questions were released they did that too. They relied on multiple test takers' memory and probably cribbed from each other. I did my private and instrument tests in that era. A little Gleim red book and Aviation Seminars weekends.
 
Brad Z said:
That is not correct. Are you sure you didn’t see sample test questions?
Click to expand...

I started in 02, and my first few tests the questions and answers on the written were EXACTLY the same as in the study guides all the way up through CFI. So while maybe the banks hadn't been released in over 20 years, the test questions hadn't changed until at least after 2006 so the questions from the banks were still being used. Either that or everyone that took the tests and turned over the questions to the guides has eidetic memory - and I find that highly improbable.
 
CerroTorre said:
As to the “you should learn the material then you wouldn’t have a problem” posters… Well, yeah. Of course. But you’re actually making my point for me. If the FAA made a decent exam, “knowing the material” would translate to passing with flying colors. The problem is that the current approach actually promotes studying to simply pass the test. The applicant is effectively forced to memorize what gets graded correct … even when it’s wrong. Some questions, if answered with real world knowledge (or logic) would get graded as incorrect. So … we grind it out and memorize.
Click to expand...
So you’re saying that “bad questions” are so common that you’d fail the test if you didn’t memorize wrong answers that the FAA thinks are right?
 
I think they're just saying, knowing the material alone will probably get you at least a passing grade, but to get a high grade (like mid to high 90s) you generally need to prepare for the test itself and its weird intricacies like some poorly worded/structured questions.

If the exam has been improved over the past few years, the calculus may not be as bad. Either way, I'm sure more improvement is warranted.
 
CerroTorre said:
I’ll be taking the Commercial written in the next few weeks and am most of the way through a test prep course at the moment. I’m trending fine, have a pretty good grasp of most of the material, have a good track record on FAA exams (short though it may be), etc. That said, the commercial test, for some reason, seems to be significantly more prone to problematic wording, “gotcha” questions, and just plain wrong answers than either the IR or PPL (where it was still a problem). My annoyance with whoever runs the testing department at the FAA is currently at an all time high. Hours of wading through the result of either incompetence or basic laziness at “work” doesn’t really help my mood :D … It’s just so bad. I picture the test writing division like the Dunder-Mifflin office of the FAA…comically bad. With some manager more interested in running around hard-core parkour-ing off water coolers and trash cans…

But whatever. It is what it is. Deal with it, just like everyone else I guess.=)

That said, my stress level (never great for exams in general) is a bit higher than it should be.

Has the FAA ever responded to criticism of the tests in a formal way? Has anyone ever had a conversation with anyone there about how poorly the tests are written? Is there an actual philosophy that tries to support the “approach” (if we can call it that)? Anyone actually know someone in the test writing office?

There are so many errors and questionably written questions, it almost seems deliberate. Which begs a lot of … um … questions.

I think most of us just roll our eyes and move on as soon as possible after taking each exam. So it ends up getting very little pressure to change. Could it not be a lot better? Maybe test actual, real world knowledge? Is it not worth it? It’s only a “gate-keeping” exam, not a real examination of knowledge? Am I barking up the wrong tree? Too many other more important things to change and improve first?

Ok, rant over. Just taking a study break.

sorry, edits done now. =)
Click to expand...

Just use Sheppard Air. Follow their strategies and you’ll make in the 90’s. Going the Gleim way is a painful process, especially after using Sheppard Air. I used them for my CFI, FOI, and CFII. Wish I knew about them earlier for all of my other writtens. I really wish they offered the A&P exams.

https://www.sheppardair.com/


Sent from my iPhone using Tapatalk
 
Nope, unlike other federal agency writtens, there doesn't ever seem to have been even an attempt at community participation in the development. The tests are archaic, poorly designed, and not even proof read. You'd often find typos, questions with no correct answer, and questions about things on portions of charts not in the supplement. It's clear the FAA didn't even have anybody attempt to proofread these things.

It's gotten even worse since someone at the FAA took kickbacks to eliminate the competition between testing centers.
 
I must also add that:
- I only found "gotcha" questions or questions where the correct answer didn't make sense in the test prep I used (ASA/Prepware)
- The majority of the questions were very similar to the bank, a few were identical
- There were a few questions on the test that I never saw in the test bank or material, i.e. related to turbo-jet airplanes

<rant>
Why do people freaking write on the freaking test supplement book (the one with the pictures)? Why?
</rant>
 
MacFlier said:
I must also add that:
- I only found "gotcha" questions or questions where the correct answer didn't make sense in the test prep I used (ASA/Prepware)
- The majority of the questions were very similar to the bank, a few were identical
- There were a few questions on the test that I never saw in the test bank or material, i.e. related to turbo-jet airplanes

<rant>
Why do people freaking write on the freaking test supplement book (the one with the pictures)? Why?
</rant>
Click to expand...

When I took my CFI exam I got quite a few Multi-Engine questions on the test. No idea why since this was the CFI-ASE
 
I took the IR written on 12/1/21, and I recognized every question I saw. I test prepped using Sheppard.
 
EdFred said:
When I took my CFI exam I got quite a few Multi-Engine questions on the test. No idea why since this was the CFI-ASE
Click to expand...

No such thing as a CFI-ASE knowledge exam, only FIA (flight instructor airplane). Not class-specific.
 
I took mine 44 years ago and from what I can remember, it was well written and I understood it all.

Either that, or it was a confusing mess and it's a miracle I passed on the first try...

I took it in the GADO office, so there is a date reference.
 
Brad Z said:
That is not correct. Are you sure you didn’t see sample test questions?
Click to expand...
They did start a process of cleaning out irrelevant questions in 2009. There were 513 questions in the IRA test bank in 2008 and in 2009 there were only 255 questions in the “Bank: (Instrument Rating) Airman Knowledge Test Question Bank”
There were even fewer in the bank in succeeding years. So it’s possible that they stopped publishing all of the questions after 2009 but didn’t change the name until 2016.

It’s possible that
They started calling them Sample Test Questions in 2016 e.g.
Instrument Rating—Airplane (IRA) Sample Questions
 
The only issue I had with the commercial written is that for some reason I couldn't come up with the answers they wanted on the performance charts. Lines too close together or something. Instructor and DPE agreed I knew how to work the problems, but we couldn't get the output the FAA wanted.
 
Yeah those performance chart questions are frustrating, especially when the testing center doesn't give you a transparency to write on. I only had 2-3 difficult ones on my commercial written (as I recall...), but I just wrote them off. The juice wasn't worth the squeeze without writing in the test supplement book, and I wasn't doing that obviously.
 
RingLaserGyroSandwich said:
Yeah those performance chart questions are frustrating, especially when the testing center doesn't give you a transparency to write on. I only had 2-3 difficult ones on my commercial written (as I recall...), but I just wrote them off. The juice wasn't worth the squeeze without writing in the test supplement book, and I wasn't doing that obviously.
Click to expand...
Yeah, same problem on my practice questions.

To an extent, it’s just the nature of using those charts - real world included. And truthfully, if my margins are so tight on takeoff/landing performance that messing up a graph line or mis-judging an interpolation runs me into that tree then maybe my decision making and risk management is a little off. =)

But it’s still an annoyance when you’re being tested and you end up with something like 1,325 feet and have an answer at 1,453 and one at 1,260. Wellp … time to round to the nearest. Except then you remember the FAA loves to round against the “rules” half the time…

It’s kinda funny actually.
 
MauleSkinner said:
So you’re saying that “bad questions” are so common that you’d fail the test if you didn’t memorize wrong answers that the FAA thinks are right?
Click to expand...
Nope. Just that if I can score higher by making my study more effective, I will. Some people don’t care if they get an 87 or a 97. Both pass.

Fair point. To each their own. I like the 97 better. :)
 
You must log in or register to reply here.
Back
Top