No, there's no *need* for him to apologize. But it appears that he has done so anyway. If he didn't want to apologize, that'd be his right, and I'd fight to defend it.
When someone says "Hey, that's offensive" you can tell them to fuck off. You can also consider if it was offensive, and...