Here's some comments from Nigel Rees who hosts a UK Radio 4 programme called "Quote Unquote". He seems to think that it relates to the singer at the end of a World Series Game (in the fifties?) but also to Opera. It looks like the jury is still out.
http://www.executive-speaker.com/rees0003.html

Here'a another couple of opinions:
"Often associated with Wagnerian opera, specifically Brunhilde’s ‘Fire Song,’ in ‘Die Walkure,’ and the fact that Wagner may seem interminable to nonaficionados"

"Originated in the United States in the 1970s. Bartlett’s ‘Familiar Quotations’ attributes the coinage to San Antonio TV sports commentator Dan Cook. Ralph Graves claims in the August 1991 issue of ‘Smithsonian’ that it has its roots in Southern proverbial lore: ‘Church ain’t out till the fat lady sings."
http://www.quinion.com/words/qa/qa-ita1.htm

"Dan Cook, a sports writer with a San Antonio newspaper, was at a baseball or football match which was drawing to a close, with the outcome seemingly inevitable.
Another sports writer said (in the way that sports writers do, always seeking the colourful phrase): "The rodeo isn't over 'till the bull riders ride" and Cook riposted with "The opera ain't over 'till the fat lady sings""

"When I was young the phrase was: "It's not over till the fat lady dies." This was in reference to operas which had plump divas who sang the main roles and who invariably died at the end of the opera. The story is that a father took his child to the opera and the bored child kept asking when it would be over and the father answered: "It isn't over till the fat lady dies." This actually makes far more sense than the present usage which doesn't make sense at all: "It's not over till the fat lady sings." This is usually the beginning of the opera and not the end at all. How do you think it got changed? Why is the present pointless usage persisting?"

http://www.shu.ac.uk/web-admin/phrases/bulletin_board/4/messages/1134.html