Helmholtz Watson said:
I understand that it is hard to make everybody happy, but again then how do you choose who you appease or not? Do you appease the majority and ignore the feeling of the minority? Or do you do something else?
There are so many ways to look at that question...
You could go with the strictly personal view: Does it benefit me more to do one thing than the other? If it causes harm to one person, but other people like it when I say that, maybe I should say it anyway because it's more beneficial to me? This is a pretty selfish view, but some people believe it.
You could go with the highly sensitive view: If one person doesn't like it, then I should never say it. This is actually much weirder than it sounds. E.g., I have an asexual friend who feels like she's hounded by people about the subject, and doesn't want to hear people discussing sex. Other people wince at even the mention of rape, even if it's a serious topic of discussion. It's not reasonable to prevent people from communicating at all.
I think of it more as a set of metrics:
Is someone here actually offended by this (which I might know from prior discussion or because they just said so)? Is someone likely to be offended? Is their offense personal or projected? Is it related to me, or to the topic in general? Do we have previous agreements about discussing offensive things? Is this a place/situation where boundaries are specifically lifted? Would not being able to use the term prevent me from communicating something of value? If so, what is the relative value of the potential harm versus the lost communication?
Or from the other side:
How offended am I at that? Did that person intend to offend me? Do they realize it offended me? Are they offending other people? If my offense related to that specific person, or what they're saying? Is there some agreement under which it's not appropriate to object? Will my objection interfere with valuable communication, banter, or just a stupid joke?
You can't always answer some of those things precisely, but if you think about those metrics, the answer is usually pretty obvious.
But then most people--including me--are occasionally assholes anyway. And this is why I disagree strongly with the OP. No matter how hard you try, you will never stamp that out--you'll just make people hate you for trying. If you're going to get along in the real world, you have to allow some things to bounce off. There is plenty of intentional offense to go around (have you *watched* any U.S. politics recently??) without getting in people's faces about things they didn't intend.