The warden could easily keep records to know when every prisoner had been to switch room.
The prisoners could not.