As a society, we like to blame objects for our personal shortcomings.
If you wanted to do things with your life, you should have. Games didn't make you do anything. Even if you were compelled to play them so much, that's your obsessive nature.
If games ruin lives, it is because those individuals chose to neglect their lives over the games. If those individuals are "young and impressionable", their parents failed them by not pulling them away and to other activities more often.
The same can be done with any hobby, interest, or activity. Like sports, music, model trains, swimming, smoking, drinking, car-building/racing, careers, education, family, religion, anything. If you become too focused and let it take up too much of your time, it is your fault. Some, if not most, of those things aren't "bad" for you, but if you miss too much work to "spend time with your family", eventually you don't have a means to support that family.
It is the same as the "gun" argument. "Guns don't kill people, people kill people", is a fairly accurate statement, but it misses the point as well. People commit crimes and violence for many reasons; social inequality, economic inequality, under-education, basic living needs not met, or a sense of entitlement. Instead of focusing trying to resolve these issues, the people, and therefore their governments, decide to focus rather on removing guns. Why? Because solving those problems would take considerable effort from everyone, but slapping down new regulations is something we've made an art form of.
In short, I suppose: if you have come to the conclusion that gaming is something you'd rather not do instead of all sorts of other things, good for you. Don't blame them for what you chose to miss out on, though.