does one need to continue doing it?

Predictably: it depends.

If the people offended are actually offended by what they think you said rather than what you actually said, it's quite important, and sometimes crucial, to straighten out the misunderstanding.