"Garderobe" comes from the French garder, to keep, + robe. So a place to keep clothes, literally. In other words, a closet. So it sounds to me like a euphemism, perhaps based in part on the size of the room. Of course in some English speaking places the room where the toilet is kept is called the "water closet" or "W.C." for short.