Like you, probably, I got a very systemizing brain and had trouble understanding this for decades. What the dictionaries say about the words does not matter - what is offensive to other people does not follow any system, but their feelings. And that means its arbitrary and you have probably no chance to know it ahead of time.

I feel your pain, tho.