The situation is more complicated and nonsensical than you think, Jazz.

For openers, English did at one time have three genders, like German, to which it was similar, but that was Old English, or Anglo-Saxon, which was spoken in England up to the Norman Conquest.

To give you an idea of what it was like: OE had 3 genders (masc., fem. & neuter), 5 cases (nominative, genetive, dative, accusative and instrumental) and 2 numbers, sing. & plural. Nouns had two full declensions (employing all the above) some of which had more than one form, and some partial ones which are mostly exceptions to the regular declensions. All this means that there were hundreds of possible forms a noun (or adjective) could take in being declined, many of which were, of course, the same. The definite article ("the" in modern English) and indefinite article (a, an in ME) were also declined. Verbs were a whole other story, even more complicated.

This is the same story with all ancient European languages in the Indo-European family.. Latin and Greek have the same 3 genders; and gender, except when applied to people or animals, has nothing to do with sex. Obviously, a word for "bull" in any language is masculine and "cow" is feminine. Beyond that, however, grammatical gender is something not well understood. C.S. Lewis, a philologist of note, alluded to this and had no explanations.

As the Western European languages developed, they took on different paths. Most of the Romance Languages, derived from Latin, dropped the neuter gender and wound up with everything being either masculine or feminine. German retained the neuter. German has a peculiarity -- any word with either of the diminutive suffixes "chen" or "lein" is automatically neuter gender. Hence a "Mädchen" (little girl) or Bübchen (little boy) or a Fräulein (young woman, literally little wife) is neuter gender and is, correctly, referred to as "it" if a pronoun is needed for the noun.

What's more, there isn't agreement on gender between the Romance Languages and others. In Italian, French, Spanish, etc. "moon" is feminine and "sun" is masculine; in German it's the other way around. Similarly, Life and Death are feminine in Romance Languages, masculine and neuter, respectively, in German. I don't know about the Slavic languages, but imagine there are the same discrepancies.

Your question is a very good one, one which I have myself often ruminated on and I would also be glad to see some explanations for the mysteries of grammatical gender.