I agree with everyone else, I don't switch fuel tanks in the pattern. I also don't switch tanks between leaving the ramp and initial climbout. I've seen some checklists phrase it "switch to a tank with fuel" which is slightly better. Really what this is all about is making sure you're not going to run out of gas with your currently selected tank.
Something nobody really mentioned is that if you inadvertently switch from a tank with fuel to one without, there probably won't be an immediate change. You have some quantity of fuel in the lines, in the gascolator, in the fuel bowl if you have a carburetor. How long will probably vary quite a bit between aircraft and power settings but you absolutely could switch to an empty tank before entering downwind and not have the engine stop until short final. Maybe even from farther out than that.
Likewise if you do switch to an empty tank(or put the selector between detents or "off" and have your engine quit it will take some time for the system to re-prime(meaning fuel getting from the tank to the engine). How long will depend on the aircraft but 20-30 seconds is not unheard of. Add in some time to process what's happening, time to figure out what to do and do it and you can absolutely end up making an unintentional power-off landing if that happens at pattern altitude.