Might it have something to do with syllables? Assuming you're using the word in a context like "____, it's time you and I had a talk..." or something, "son" is a lot easier to say than "daughter"...and it sounds less formal too. But then again, who am I to know?