Or perhaps it is a reference to the genitals, which are "south" of the conversational part of the body....might just mean the conversation has got to the "unmentionables" pretty quickly?

People also refer to oral sex as "going south" - since we're being risque an' all on this thread....