A game's sound design is critical for to develop a sense of atmosphere, and that includes silence.
Music is only part of the equation, if easily the most recognizable part. A fast-paced tune can give a sense of urgency or desperation to a situation, a slow one can impart sadness & despair, crescendos and decrescendos (rising and falling rhythms respectively) can reflect how the player is supposed to be feeling (hopeful or depressed, respectively). Moments where concentration are needed can have subtle consistent beats. Another common trick is for the music to be dissonant to the situation (the lyrics don't have to match the rhythm either), which makes the scene which uses it all the more poignant; comedy is seldom used for this, but certainly despair & horror make good use of the technique.
Sound design plays a huge role in any atmospheric game, as it helps create and maintain that sensation; music is just the most common tool for doing so.
---
But yes, atmospheric sound design goes beyond just music; sound effects, to convey what's going on, can play a surprisingly large role. Footsteps (be it belonging to the player or NPCs), voice work, anything. Screams & shrieks are very common tools for atmospheric games, which almost always raise the tension or suspence of a scene; except when clearly used for comedy (i.e.: big guy screams like a little girl), they're innately horrific and/or terrifying and say something bad has happened.
Which brings me to the role of silence. If you pay close attention, you may realize that the real world is seldom truly silent. When things go deathly quite, especially if for longer than a couple of seconds (a "beat"), people start to get nervous very quickly. Either it means you're alone (and because humans are innately social beings, not a pleasant sensation for most)... or something's very wrong (particularly when things should not be completely silent).
Dark Souls is a game without a lot of music, but there's no question it has some outstanding sound design where silence plays a strong role. The lack of music heavily implies that you're alone in the game, and most of the time the only sounds are that of movement. It also forces you to concentrate on the task at hand; there's no music to distract you when you move through the world, meaning you'll notice the moment something changes. Music is used for boss fights, and is clearly used to convey the surrounding circumstances rather than the pace of the battle; dissonance is used at least twice (including the final boss battle), slow and somewhat sad music for two notably quick-moving bosses.
---
Music, as a general rule, adds to a game's atmosphere. However, it would be more accurate to say that great sound design always adds to a game's atmosphere; music is a part of that, and it's absence is just as vital as it's presence.
(Also, there's something about playing Fallout 3, absolutely slaughtering ghouls with lincoln's repeater as "I don't wanna set the world on fire" plays from your Pipboy) 
