Generally, I don't. If it's only a single wasp or a few of them looking for food, I'll gladly share my apples, pears and whatever is on hand with them. I also usually cut away a bit of steak for them to tuck into so they don't feel the need to get on our plates, which has worked just fine all my life.
If they were to build their home inside our walls, that's overstepping and it will be handled with smoke first. If they don't get the signal right and start looking for a new home, I'll gladly tear the wall down, set it on fire, nuke it from orbit and start from scratch. Don't get me wrong - I love and adore wasps, but they're just not too cool around pets and kids. Very cool to police the garden, though. They're like tiny, flying Rottweilers. I like that.