>I recall it being used strictly as an epithet aimed at people.

are you possibly thinking of schmuck, which while similar, is an other word entirely?