It's sad to think of silence as a rare commodity in games, but it can indeed change the atmosphere of a game. Case in point, the original Tomb Raider. Apparently, the playstation version had subtle but present level music, while the PC version mostly only had ambient tracks, which included lots of silence, wind blowing, water echo, etc. The experience described by players of either version can be different, with the PC crowd tending towards more isolation, tension and mystery than the playstation crowd.
In stealth games, Mark of the Ninja recently takes silence to a greater visual extreme. Every noise is graphically represented by a circle marking the radius up to where the sound of feet, birds flapping, gong clashes, glass breaking, dart clinging, etc, will propagate. It serves the gameplay by making sound an integral part of how to execute an elegant distraction/sneaking/attack plan.