I've only heard it used one time with any frequency. When I was a teenager, a black woman at the end of our building always referred to her kids as tarbabies. I don't think she meant to convey any evil intent by it.
At the time, I had never heard the word used by anyone before. I had assumed she just made it up on the spot. I'm not sure whether it would occur to me to use this on other people, but I've thought several times about referring to my youngest daughter in this way, as she tends to get very dark during the summer.
OTOH, there are attempts to silence people from saying things on the basis that saying those things in those particular ways is necessarily racist. For example, if a white person were to talk about "asian mothers," she would almost certainly be met with ridicule and scorn. Any time my kids are talking with other asian or half-asian kids, an inevitable topic of conversation involving mutual mocking and commiseration, is their experiences with their asian mothers. If you look on the web, you might even find a few websites, including videos, of asian kids poking fun of their parents.