I encountered this phrase twice yesterday: once on the radio (I can't remember the context; wasn't paying enough attention), and again in the NYT Book Review section, where Jay McInerney uses the expression as an aside: "Sorry, I tried, but I couldn't ignore the elephant in the room."

Yesterday was the first time I'd heard/read the expression. Is anyone out there familiar with it? What exactly does it mean?