Stereotypes are not inherently evil unless there isn't any truth to them at all, (i.e. Asians having webbed toes or something ridiculous like that). There are definitely racist ones that aren't completely false too, I'm sure we can all think of a bunch, (Black people being better at basketball than other races for example). The important thing is that even if one buys into these stereotypes somewhat they are generalizations and are never true in every instance, and as long as you keep this in mind you can recognize that every individual must be judged individually, to be prejudice, if we merely go by the definition of the word, is merely to have preconceived notions coming in to a situation, which is not racist as long as one realizes that those notions may be wrong, and the attributes in question must be verified by the particular situation of that person. Say, for instance you had an interview with an Asian candidate for an engineering job, and you think that Asians tend to be good at math. You can't verify that notion by looking at the candidate and saying to yourself, "Yup he's definitely Asian and therefore good at math," that would be begging the question; you have to actually look up his grades in math courses and his math scores on standardized tests. But you can't make a general claim about a group without it offending someone, even if you have proof. Even if you cited statistics that black people tend to be better on average than whites at athletic activities, you could still very well offend someone. But in the case where your claim is backed up with evidence, people cannot complain, even if they are offended. After all, if objective facts about the world offend you then that's tough cause the world ain't changing itself to make you happy.
But the label "Gamer" is different because you can change it. People cannot change their ethnicity or cultural background, but no one is inherently a gamer. There are two main connotations for the word. There is the economic label, which is equivalent to the term moviegoer, and is not about the lifestyle of those people but merely about what they buy. The second, however, is about lifestyle. There is a definite difference in the information conveyed between the statement "I play videogames" and "I am a gamer." Yahtzee claims that the only real difference between the two is some sort of negative connotation and stereotypic image is attached to the latter. However, I believe that the aforementioned image and connotation result from a piece of information that is only conveyed by the term gamer: level of interest. A gamer is different from a regular person who just plays videogames in as much as videogames occupy much of his interest beyond being an occasional pastime. It is comparable to the difference between a person who watches Star Trek occasionally and a Trekie. Trekie has negative images and connotations associated with it as well, but the defining characteristic of Trekies is that they would admit that they are Trekies and would only falsely deny it in order to avoid embarrassment. You see the difference between this type of label and racial stereotypes is that the label itself is applied through the behavior it predicts. A person can deny that they are a Trekie by listing all sorts of things; they haven't seen all the episodes, they don't have all the stuff on dvd, they can't tell you the names of the individual episodes, they don't have any Star Trek paraphernalia, they don't dress up as ST characters and go to ST conventions, etc, etc. The same with gamers, if someone wants to deny that they're a gamer they can likewise list all sorts of reasons; they only have one console, they play games for online matchmaking and don't care about story all that much, they have other interests outside of games that they value over games, they don't care about special edition boxes, they don't care about completing all the achievements in the game, they only care about difficultly level insofar as it makes the game challenging but still fun, they're willing to use cheats to get through parts they're stuck on, they don't go onto game chat-rooms, etc, etc, you get the point. Neither of the two lists above were meant to be exhaustive, and frankly both the terms are rather vague such that it may not be clear what the dividing line exactly is. But the general dividing line is the level of importance games/Trek plays in one's life. Despite its vagueness the term "gamer" does have meaning insomuch as it, at least in its positive attribution to someone, tells us that videogames play an important part in that person's life. The chances are that if someone is willing to label themselves as a gamer, it does at least convey that videogames are a very important pastime for that person. As with any other label it might not be properly applied (the person could just be lying to impress someone who they thought was a gamer), but in contrast to racial stereotypes, if one rejects that they have the qualities of the label then one rejects the label itself. An Asian could say that he sucks at math, and would obviously not be denying that he is Asian, whereas if someone denies the qualities of a gamer then they are not a gamer.
My point is, only gamers should call themselves gamers. If you feel like just saying that you play videogames doesn't express the level of your interest in videogames then you should probably call yourself a gamer, although you still have the right to deny any of the accidental qualities associated with the term. But if videogames are just one of your many hobbies and do not take a special role in the hierarchy of your interests then calling yourself a gamer is rather dubious.