Since it puzzles people why some people can say the N word with no repercussions and others can't: because there is an automatic association with centuries of abuse and oppression, white people, and the n word. You can whine about it all you like but it still exists. You can make rape jokes around a rape survivor, and when they get freaked out say "come on, it's only words" but you're still the asshole.
You won't get arrested simply for using racial slurs, there is no legal setup for that because it is outside of legal jurisdiction. However, it is well within SOCIAL jurisdiction and ultimately any rules (not laws, individual organizational rules) that arise around political correctness exist so that people don't have to deal with that bullshit in their schools and workplaces. Any social repercussions for using offensive words exist because people don't want to deal with that when they're just chilling with their friends. If someone referred to me by the slang term for Jew, it would imply a level of hatred, threat and disrespect that I don't think it's overly sensitive of me to be bothered by.
For instance, I don't swear around young kids and older folks, because it upsets them or in the former case their parents. This has nothing to do with me being oppressed by their (or their parents) social mores and everything to do with me having respect for the people around me. If I felt I was being truly oppressed or made to hide something important about myself or my beliefs, I would argue the point, but some things just aren't worth fighting for, and my right to say fuck without consequences isn't one of them.
Very few people actually have a problem with the word Cracker. Hell, I don't have a problem with saying it, though I am white which skews the data a little. It would seem odd as an insult, almost quaint. Yet if I knew people who were bothered by it, I would make a point of not using it around them. Still as a pure numbers game, much much more people would be upset if you called them the n word than the c word.
(assuming you meant cracker and not ****. Because I'm not really sure why you'd compare the two otherwise...)