I don't know if you can outgrow them entirely. More likely, as people get older, they might find their tastes shifting to one specific genre, or time period. It's like my Dad doesn't listen to FM radio; he listens to AM news radio. It doesn't mean he outgrew radio as he grew up - its place in his life just changed.
The way I think of people outgrowing games is more in the sense of not keeping up with the trends due to not having the time to play video games (work, family, college, whatever gets in the way) and progressively becoming removed from the improvements in technology after not playing for a few years. This even happened to me in highschool. For a few years, I went through a period of not buying any new games. Maybe getting one new one for my birthday.
I won't go into reasons for it, but, for a while, I really wasn't keeping up with gaming. With maybe a few exceptions, I was playing PS2/PS1 titles from 2002 or earlier until about 2006, when I got an Xbox 360. The gaming industry, and gaming as a whole had changed so much while I was busy with other stuff that it was almost overwhelming. So, if that could happen to me while I was still a teenager, I imagine it can be pretty alienating for someone in their fifties like my Dad, very rarely has the time to pick up new games (maybe one new PC game every 2-3 years, if he's not busy). He still plays games, but it will be the same title, or an old title. I'm pretty sure I've been listening to him intermittently play the same shooter for more than five years now.
So, yeah, it's not necessarily a case of outgrowing gaming so much as maybe having the gaming industry outpace you as an individual, or becoming out of touch with the evolving gaming culture, maybe. More often than not, though, people who grew up on games, or at least had gaming as an important pastime, will still pick up a game they loved from years ago and keep playing it, the same way they might watch an old TV show or movie, or read a book they read in high school again.
They may not be buying new titles, but that doesn't mean they've outgrown gaming. It just means they've had a lot of practice at becoming really good at a handful of specific games.