For me, assuming it's a chav, (a feral, arrogant, borderline criminal wigga, for the non brits out there), and I will, because they're the type who need beating down...
'Baseball caps are like pubic hair...you can remove it but there's always a c*** underneath.'
(I know statistically , its only about half the time, but why let fact ruin a decent line.)
EDIT: depends how you interpret the stars..looks like I'm covered!
Btw, nothing against people outside the UK wearing em, where you may have baseball...or sun.
When you're just wearing it because it keeps your face out of security cameras when you're out shoplifting, or you're *shudder* wearing two at once (why??), or wearing one under your hoodie, then you need a slap.