>If the majority of batters are right-handed, then most balls would be hit toward left field. Why the negative connotation?

you are right(!) -- right field was akin to banishment in little league (I know this), and even in the majors it is where you will usually find the weakest defensive player (outside of the DH ). William Safire poses two theories (in his book "I Stand Corrected"): 1) it was an insult hurled at kids stupid enough to buy left field seats at Yankee Stadium in the days when Babe Ruth (great hitter, poor fielder) patrolled right field for the Yanks -- but weren't most of his homeruns hit to left? 2) at the old West Side Park in Chicago there was a mental hospital in back of left field.

as to southpaw, the baseball connection certainly seems plausible, but I have seen it said that the earliest known *written citation was in reference to a left-handed boxer.
http://www.word-detective.com/093098.html