I think you've just got to put it all down to usage. Originally, most of the castles didn't have anything resembling toilets, but they did have storage cupboards built into the walls. There were latrines for most people, usually just longdrops within the bailey with a plank which fitted all comers. This was shifted as the holes, um, filled up. The fortunate occupants of the keep used buckets and the "lucky" servants got to carry them out and empty them into the latrine in the morning (for the lords and ladies). The men-at-arms and servants within the keep had to shift for themselves - perhaps the bucket brigade again.
Later on, castles were designed slightly differently so that the garderobes were extended out from the walls and became latrines. In fact, Edward I, from memory, specified that this should be so, and he was the champion castle-builder. Usually they were on the outside walls and in many castles the strategic vents were located directly over the moat. I don't know what happened where there was no moat, presumably it just hit the ground and ... er ... decomposed over time.
Some sieges were broken when dysentery and probably hepatitis broke out within the beseiged castle. It isn't difficult to speculate that the sanitary arrangements within the castle walls, primitive at the best of times, became overloaded and broke down completely during sieges.
So the alcoves kept their name - garderobe - while the usage changed or, at least, some garderobes may have retained their original function while others became privies. That's all there is to it, really.
