I think you should replace "casual" with "people with lives".
Honestly, I've been a gamer since I was a little kid. I used to spend entire days playing video games and I still spend much of my free time doing so. But I'm a grown man now and most days I don't have time to run a 4 hour instance. I have to go to work, clean the house, make supper, and all that other grown-up stuff. Long gaming sessions are almost non-existent, so I can't dedicate myself to scheduled raids, large instances, and takes-half-an-hour-to-walk-there quests.
But I still like to game and I still like to play with people. That's why MMOs, despite how much I have hated nearly every one I've tried (Can we please do something other than kill waves of baddies? It's getting old.), are still interesting.
The other problem is what John Gabriel termed The Greater Internet Fuckwad Theory. Put a thousand people together in a room with no real identity and no fear of recourse and see what comes out of it: homophobia, racial epithets, and general ass-hattery. The hardest quest in most MMOs is finding a group of 20 amicable, amenable friends. Unless you come into it with your own huge group of friends, you may as well get used to playing alone, because it's not worth the effort to try finding a group most days.
Lastly, most games really punish you for working in groups. Sure, it's the only way to do a 25 man instance (which is where all the good loot is), but during your three month crawl from level 1 to 80, you'll be penalized heavily for having a friend along: you'll each get less than half of the XP you would normally get, plus you'll blow through the limited number of mobs long before the quest is over and spend 20 minutes waiting for them to respawn. At least you'll have somebody to talk to while you're grinding through the meaningless, unchallenging quests.
So there's a good reason many people play solo, and it has less to do with "casual" gamers infesting our "hardcore" games and more to do with the limitations of adult life, the social skills of your average gamer, and the broken systems used in nearly every MMO to date.
If you want a real multiplayer game, go play Team Fortress 2. Not only do you get to interact with that massive group of people, you can kill them if they piss you off and you can ban them if they are being asshats.