What makes me so sure is the circumstance. See more below. And yes, old people in Florida aren't known for their diplomatic talent, they are 100% tone deaf. And I'm confident you know everything I have to say about Goldwater and all that, so I'll let it slide cause we don't need to re-dredge the southern strategy debate every thread.
It's not the same when it's a response. If this guy zipped around in a golf cart yelling white power unprovoked, I'd be inclined to take him at his word. If he then said it was just a joke, that doesn't change my perception. But it was an instigated statement, it was a response to being accused of being in the klan. Saying something offensive out of nowhere and then calling it a joke is crap behavior, saying something offensive because someone is accusing you of being that offensive thing isn't the same, that line is at the accuser's expense.
If I were to come in and start accusing the communists among us of being genocidal maniacs, and someone responded "Yeah, you got it, bring on the gulags," I'm gonna say they're also taking the piss like this guy was. If, however, someone volunteers this line:
I'm inclined to take them at their word.