I have no problem with the term to be honest. The complaint that Yahtzee seems to be making is that the label is accurate, and that's not fair to the non-nerd population that also play games and don't want to be lumped in with them.
I do understand where he's coming from though. Truthfully the term "gamer" for most of my life was mostly spent referring to people who played paper and pencil RPGs. For a long time it was something of a badge of honor and empowerment because you were dealing with a lot of very smart people. The industry started dumbing things down and trying to go increasingly mainstream. You wound up getting people involved who were not "smart nerds" but simply social rejects, this dragged things down. Kids were being brought into the hobby by differant marketing, and you even saw shifts from a situation where kids were a minority in the hobby (and mentioned in say the Forum of Dragon Magazine as one of the issues), to where they made up the majority of PnP gamers (and this of course also became a topic of discussion in things like the Forum of Dragon Magazine). At some point we got away from a collegiate level hobby where you might have one player out of a thousand (in the then much smaller community) who was a kid able to do it and see respect, to a situation where PnP RPGs became a stereotypical refuge for especially reject-worthy high school freshmen who couldn't even be redeemed by their relative intelligence. Compare say the current edition D&D books to the first edition of AD&D and the language that was used. That says a lot about the people that played and the requirements to understand one of these games.
As someone who rode the PnP bus as it went down in flames, watching as things changed, I can say that there was a lot of resentment similar to what Yahtzee seems to be feeling. A desire to differentiate yourself from the guys who got into the hobby when it expanded. I started gaming young, and had a lot of problems (as I've mentioned before), but I will say that I have met gamers that can almost inflict physical pain on me just by hearing them talk about gaming, and as much of a wreck as I am can sit there and go "that's what I've been associated with?".
It might be a passing distinction, but someone with a big huge brain (even if they are not smart in any practical way), who tends to be slobbish and socially awkward has some redeeming issues because he's ridiculously smart and knows all kinds of stuff. In many cases being like that because they are operating on a somewhat differant level from the norm. On the other hand someone who is like that and dumb, or simply of average intelligence, with no real base of knowlege, practical or otherwise, is simply a slob. To a lot of those people gaming is a way to try and justify themselves by latching on to a similar seeming group of people.
This is similar when it comes to video gaming, the current use for "gamer" in that the industry has also dumbed down, and gone mainstream. You have bucketloads of people streaming into the hobby, and acting like there is something to be proud of in being a "gamer" when any pride that could have come from that was washed away in lowering the bar to the point where they could be one.
To sort of put things into perspective, let's look at my favorite genere: RPGs. I love deep RPGs with tons of cutomization, items, stats, skills, and massive networks of spreadsheet like menus and sub-menus. All the stuff Yahtzee hates ironically given this message. I get a ridiculous amount of enjoyment out of tweaking variables and then watching what happens when I hit the "attack" button or whatever. I also like large, complex game worlds that take a lot of time and effort to get everything out of. However, today look at how most RPGs are simplified. Most of the games of the genere get by as being "challenging" or "hardcore" purely on length. Sometimes with an epic RPG I feel like the biggest challenge is being able to stick with it for 100+ hours of gameplay which no sane person would endure, with little going on to justify that. I mean, heck half the RPGs today basically play themselves (like the last two Final Fantasy games), and in a lot of cases the stats aren't much more complicated than playing "Rock Paper Scissors" where you can see what your opponent is going to throw ahead of time (ie use X, to counter Y... repete ad infinium), with that being not an aspect of combat, but the entire principle it's based on. Rock, Paper, Scissors never being a really intellectual passtime.
Truthfully I will also say that I think the term "gamer" exists to exclude normal guys like Yahtzee mentions. I mean that's because they are normal guys, not hobbyists. Above and beyond the quality of gamers the point is that those who "wear" that label good or bad are not involved in gaming as a passing amusement. The label "gamer" is used by those who employ it to diffentiate themselves from the mainstream. Right now you have divisions within the "gamer community" looking for sub labels.
At any rate, I guess all of this amounts to yet another expected "I am gamer, here me whine" rant.
I also can't complain too much because face it, I'm a broken unit physically and mentally. I fit every negative stereotype about gamers. I'm fat, have glasses, live at home, and others assorted things. I do however shower daily and don't smell of anything, and am clean shaven whenever I go out of the house.