My friends and I will call ourselves gamers because, well, we game. Its our hobby. I swim and sing in a choir as well, but the majority of our time is spent playing, modding or researching games.
Sure its got a negative connotation in some groups, but that all depends on your frame of reference. In my life, the groups that it has a negative connotation in are the groups that have a negative connotation in my circle; 'Normal' people [Vague, yes, but those in my friend group understand what you're talking about, and its the most appropriate label that isn't somewhat insulting], sports nuts and politicians.
Yeah, its got a negative connotation in those groups. I'm not in those groups, so honestly why should I give a f***? They're not a big or important part of my life, and I don't want them to be, so why should I care if they don't like me because of my hobby?
Also, I realise that may come across as somewhat hypocritical, but those groups generally receive the negative connotation thanks to their attitude towards gamers. If one of them comes along who is open minded, I'm all good to be friends with them - and I often am.
I personally don't see anything wrong with the term. Its an extremely vague descriptor taken literally, but add in context and you get a more narrowed term that describes a group of people. Yeah, people can't be summed up by one word describing their main hobby, but that's not my problem. Ignorant people will be ignorant and discriminate regardless of whether you openly state that is your hobby by calling yourself a gamer or not, and they're, IMO, not worth wasting time to try and please.