I argue silence in most games is a terrible idea. Even if the game is silent, the loud noise of an x-box and/or computer, and tv will become more apparent taking the player out of the experience. That's not including sounds from every day life. If one lives anywhere near other people, then one will hear things like music or lawnmowers. If one lives alone, one will be distracted by the sounds of nature.
For me, the moment sound drops out, I loose all sense of tension or solace.
Moments of silence can be indeed be wondrous in an otherwise frantic game. Open world games do seem to pull this off the most, although some others like The Last of Us do it well too.
I had a similar experience to what you had in GTA in Far Cry 3. When I'd get away from the roads and towns and observe nature, whether it was from the top of a radio tower or dangling from a hang glider. Pure bliss.
It's sad to think of silence as a rare commodity in games, but it can indeed change the atmosphere of a game. Case in point, the original Tomb Raider. Apparently, the playstation version had subtle but present level music, while the PC version mostly only had ambient tracks, which included lots of silence, wind blowing, water echo, etc. The experience described by players of either version can be different, with the PC crowd tending towards more isolation, tension and mystery than the playstation crowd.
In stealth games, Mark of the Ninja recently takes silence to a greater visual extreme. Every noise is graphically represented by a circle marking the radius up to where the sound of feet, birds flapping, gong clashes, glass breaking, dart clinging, etc, will propagate. It serves the gameplay by making sound an integral part of how to execute an elegant distraction/sneaking/attack plan.
I forgot about the early Tomb Raider! I never played it on the Playstation, but it was most definitely strangely quiet on the PC. It fit the concept of exploring long-abandoned spaces so well.
Mark of the Ninja is also an interesting game with regards to sound design. That giant sound signifier can become just as dangerous as it is helpful, and the fusion of sight/sound representation is such an inspired way to map ninja senses.