I recall hearing tapes of a Ground controller who instructed a pilot to taxi somewhere and shut down the engine, because he suspected the pilot was drunk. Turns out the pilot was drunk, and was arrested, and the controller won an award for the save. The controller didn't have to explain himself on the radio in the moment. (I think I heard this on Aviation NewsTalk or similar. Too lazy to look it up right now...)
Well, there was that looooong pause in between "because of" and "you", during which I imagine the controller was searching for the right words and just not finding them. Or perhaps finding words ("because you're becoming a potential safety hazard") but then deciding they would not be appropriate for the radio. If that were the case, it seem's he'd be justified in not clearing for the option.