During the thirties when many Canadians worked in New England, using the word "bum" in a bar could start a brawl. But I never heard Americans use it as a serious insult. It was innocuous enough that the Brooklyn Dodgers were frequently referred to in the newspapers as "Da Bums".