Reminds me of Elizabeth Taylor's 7th husband. She just couldn't find a good man.
In other words, maybe it's not them. Maybe it's you. Or more specifically, maybe that instruction at that time is not conducive to good comprehension. Pilots anticipate a certain sequence of tasks in a certain order, and mentally rehearse their actions so they don't screw it up. If you want them to deviate from what they expect, perhaps you need to do more to call their attention to the deviation.
Suggestions:
1. Separate the transmissions. "Remain this frequency during taxi", wait for acknowledgement, then give taxi instructions.
2. More words. "Remain on tower frequency, do not change to ground".
3. Add the instruction to your ATIS or AWOS when applicable.