such a person is a bane, a pest, a bête noire, a bugbear, a curse... and speaking of a curse, could be personified (so to speak) by Joe Bfstplk, the Al Capp cartoon character who walked around with a storm cloud over his head and was a total jinx but otherwise was pretty likable.