Ah, but they CAN effectively detain you. I have heard, a couple of times, TSA screeners saying "do you want to fly today" or "do you want to make your flight because I can be sure you don't".
The TSA policy is "no one may leave once we start screening - even to the 'unsecure' area - until the screening process is complete". So, you can refuse to answer the questions, the screening process never completes, and you are NOT free to either make your flight NOR leave the screening area. According to the TSA, they decide when screening is done. So, you are, effectively, detained until the TSA lets you go. Now, as a practical matter, you can demand a supervisor and law enforcement officer. Doing so is at your peril because it becomes your word against theirs. And since the are "The Man", they are automitically deemed to be more credible.
They took exception at BWI to an aviation headset and aviation handheld radio one time when I was going to retrieve the plane from annual. I got the snarky "do you want to make your flight" speech when I asked them not to rip the lining in my suitcase trying to open it (I offered to show them, and the guy got real defensive and snarky stating that he knew how to open it). He then proceeded to tear the lining because he really didn't know how to open it. I demanded a supervisor who appeared and refused to give me his name, her name, or the name of her supe. She even claimed not to know who the FSD was for BWI. I wrote complaint letters to the TSA, the local FSD (a little research is a good thing), and the airport manager. None of them even bothered to send a cockroach letter. The result of my calling the supe was a 15+ minute delay while I was "punished" by being sent for secondary inspection, and all the info from my boarding pass was copied down (I refused to let them take the address from my DL). Fortunately, I was 2 hours early for my flight (tried to get the earlier one, but nada thanks to these clowns).
The problem is that there is no accountability. There is no punishment for a law enforcement officer (in many places) that abuses that "reasonable cause" exemption. There is no accountability for TSA screeners that abuse their positions. There is little accountability for gate agents or flight crew that abuse their authority in calling something a "security" issue when it's not (note: I know the call is tough sometimes, but there ARE cases where crew/agents have abused the authority, even one who tried to confiscate a camera - calling it a 'security' issue - that someone was using to take pictures of their friends going on vacation). Instead, each may or may not be counseled then complimented for trying very hard to stop a potential terrorist (see "zero tolerance" policy thread in Spin Zone).