No, for the same reason I very rarely listen to music when gaming. It makes it harder to pay attention to what is going on.
If it was the original Fallout or some other turn based game then I suppose I wouldn't mind. Normally the games I play are "real time" though and games such as that often require paying attention to the games sounds, so outside noises are just a distraction.
Most multiplayer games for example use sounds as a means of gauging how close the enemy is nearby. Team Fortress 2 has turret noises, Gears of War and Natural Selection 2 have footsteps and so on. Any external noises make those harder to pick up so I don't have any other audio playing.
I normally listen to music when playing FTL seeing as the game only requires visual cues to get by. It does have sounds to help, but they are not required considering the game can be paused at any moment to issue commands and take stock of the situation.
As for audio books specifically, I'd just find that even more of a distraction than music, because it's telling you a story while you are playing a game that has a story. It's like reading while watching a film, I honestly couldn't see the appeal. Then again, I don't listen to audio books anyway, I much prefer reading.