Yeah, I know "outhouse" is a legitimate word. Perhaps "correct" was too strong a term, but in the context of that particular phrase, in approximately 40 years of my hearing/using it, the polite form has always been used with a touch of the "I'm toning this down" inflexion. Possibly it was originally "outhouse" and got changed by folk of a more robust vocabulary.

I stand by only ever having heard the phrase used in reference to a well-muscled personage, though.