wog sounds racist because it is racist:

slang.

[Origin uncertain: often said to be an acronym, but none of the many suggested etymologies is satisfactorily supported by the evidence.]

1. A vulgarly offensive name for a foreigner, esp. one of Arab extraction.

1929 F. C. BOWEN Sea Slang 153 Wogs, lower class Babu shipping clerks on the Indian coast. 1932 R. J. P. HEWISON Essay on Oxford 5 And here the Ethiop ranks, the wogs, we spy. 1937 F. STARK Baghdad Sketches 90 When I return, Nasir fixed me with real malignity in his little placid eyes. ‘I knew she wanted me to go,’ he said. ‘I could see what she was thinking. They call us wogs.’ 1942 C. HOLLINGWORTH German Just behind Me xiii. 258 King Zog Was always considered a bit of a Wog, Until Mussolini quite recently Behaved so indecently. 1944 [see COME v. 39e]. 1955 E. WAUGH Officers & Gentlemen II. 323 He turned up in western Abyssinia leading a group of wogs. 1958 Times Lit. Suppl. 11 Apr. p. vi/3 We have travelled some distance from the days when Wogs began at Calais. 1965 [see COMMIE]. 1982 J. SAVARIN Water Hole I. iv. 42 He hated Arabs... They were all wogs to him.
OED2

earlier than the known wog citations, Joyce used the word "wogger" in Ulysses, in one of his run-on chapters; but who knows what he meant.

AHD and M-W guess that it's short for golliwog.
http://www.bartleby.com/61/66/G0186600.html