Online Player Skill Level - Bell Curve

[L.B. Jeffries]

I came across a weird observation last night on Xbox Live and I was wondering what other people made of it. I was getting into an argument about Call of Duty 4 vs. World at War when someone pointed out that World at War is funner to play online because not that many people have mastered the game. That reminded me of the Endo's Subspace article last week about how the game culturally started to slow down because the beginners stood no chance against the pro's. The connection is pretty simple after that: at some point people stop liking a game because the online component is too difficult to get into.

I was wondering about the timeline on all this when I remembered an article from way back about another bell curve in video games: the Hype Backlash. Leigh Alexander did this chart where she showed that it took about 4 months after a game's release before people would stop praising it and start bitching about it.

So...is that how long it takes for the new players to reach their peak and for the developed skill of current players to start driving away new customers? I'm brainstorming on something right now and I wanted to hear observations from people. Has anyone else noticed something like this?

 

[Hunde Des Krieg]

I'm pretty hype backlash stuff is like a fad, you know it's trendy, so I wouldn't read too much into it, but player skill should follow a bell shaped curve, it would be wierd if it didn't.

 

[black lincon]

Happened to me in gears one, a few of my friends were amazing at the game and got me to join in the fun, unfortunately I was god awful. However, in gears 2 when all the maps changed up and people had to form new strategies, I managed to make myself feel at home.

 

[Drakstern]

I'm pretty hype backlash stuff is like a fad, you know it's trendy, so I wouldn't read too much into it, but player skill should follow a bell shaped curve, it would be wierd if it didn't.

It's more likely to be skewed in one direction or the other.

In a game with a developed and stable community, it should be skewing to the right, as fewer new players are coming in and those that ARE in are getting better(hopefully).

In a quickly growing game, it should be skewed to the left, as new players are joining and having to learn the ropes.

Of course, you'll always have people that just aren't that good at the game, no matter how much they practice, and 'naturals' throwing off the assumptions, but I wouldn't expect a generally normal distribution from skill levels for most games.

 

[MintyFreshBreathGuy]

I've noticed this curve as well. It's more fun with a balanced out game (Team Fortress 2). I did notice and I thinks its one big pain of the neck I can't live with. So from now on I'm sticking to PC TF2.

 

[Varchld]

I noticed how it can be annoying to be a poor player for me. Every time I begin to play CoD4 or something online it usually takes me a whole match to get back in the swing of things. It reminds me not to ever think i'm good at it
There's usually one or two better then me which helps too.

To answer your actual questions at the end, some people simply can't get much better at a certain genre of game, others are naturals and others still can learn the dynamics. CoD4 was so much easier for me after I learnt where to take cover, which angles to watch and which gun for which level.
How long it takes to get sick of a game depends on many different factors and online opponents skill is only one of them, and a bit random too because it depends which games you end up entering.

I think games go on much longer before the "skill" of players gets so high that it starts to kill the fun, more then 4 months for most imo.

And if all else fails, play drunk.

 

[nimrandir]

Rather than the normal probability distribution (with its annoying little left tail), perhaps the data is better modeled by a chi-square [http://en.wikipedia.org/wiki/Chi-square_distribution] distribution?

 

[CoziestPigeon]

Funner isn't a word.

 

[51gunner]

An interesting theory, but difficult to prove. I'd imagine part of the dropoff in new players to a game after a certain amount of time is also due to something new coming out, and they go off to play that, leaving only hardcore fans behind. The remaining hardcore fans then mercilessly dominate any new players, so that half of your graph is accurate.

I'm questioning the first half though, with a game's launch you have an instantaneous surge of new players, maintained for a couple days, then dropping.

 

[LumiHawke]

Whenever a game hits the market online modes are most fun with in the first few months of the games life time. Before people learn all the dirty tricks and have the maps memorized.

I played Wolfenstein ET for nearly a year, if I go back to it now I know from experiances I'm not pure cannon fodder, people have changed the game so much with modifcations that it just isn't the same game before, old tactics are useless.

 

[Lvl 64 Klutz]

Happened to me in gears one, a few of my friends were amazing at the game and got me to join in the fun, unfortunately I was god awful. However, in gears 2 when all the maps changed up and people had to form new strategies, I managed to make myself feel at home.

I had a very similar experience with Battlefield 2 followed by 2142

 

[crimsondynamics]

That's not a bell curve.

Other than that, the observation is accurate. If you don't jump into a game from its release, you will be left behind in a matter of days, not weeks.

You certainly can stick it out and be slaughtered every step of the way, but eventually you will improve and be competent enough to remain competitive. Stop playing for a period of time though, and you will have to climb up again.

Given enough time however, unless you are extremely adept (or incompetent) everyone's skills eventually level out. The time it takes to level out varies with different gamers as we all have different skills and capacity to learn something new.

Counter-Strike has its share of exploits but when they released the Source version many of those exploits were eliminated, which helped level the playing field - for a short while, at least.

South Korea loves their StarCraft, but ever since Blizzard announced the sequel the amount of griping from serious / professional Korean gamers was made evident. Obviously, these "elite" players are afraid their skills, mastered over a decade of dedicated gameplay, will be rendered useless.

That's one of my main gripes against multiplayer in general: I don't have the time to invest in playing several hours a day, and I normally like to mix up my games instead of dedicating my time to any single title, so I tend to shy away from multiplayer-only games because I know it will be a challenge to be, and remain, competitive.

Multiplayer games should have mandatory matchmaking systems where you have the option of being paired up with other players in your own skill range, not against people who live and breathe the same game day in and day out.

 

[L.B. Jeffries]

Whew, you guys don't screw around. Thanks for the help, I'll adjust accordingly.

One of those variables, new players, is really easy to calculate using Xbox Live stats. How would you quantify player skill? The ratio of kills to deaths? Number of games won? That wouldn't work for every game but for your basic FPS or run 'n' gun.

There has to be a way to boil down the skill of a player into a number, deaths divided by kills?

 

[Varchld]

One of those variables, new players, is really easy to calculate using Xbox Live stats. How would you quantify player skill? The ratio of kills to deaths? Number of games won? That wouldn't work for every game but for your basic FPS or run 'n' gun.

There has to be a way to boil down the skill of a player into a number, deaths divided by kills?

Not that easy imo. I'm pretty good sometimes but get reckless and die a lot at the same time. I could be super accurate and be the best in the room but just have bad luck. I've got 32000 kills, 20000 deaths and i'd consider it a pretty bad ratio, though often i'll end up in the top few players for a room. (that's mostly hypothetical since i haven't played for months >< )
And then there's pugs vs clan matches skewing stats even more. A good player in a pug team match will have worse stats then an average player in a clan team match.

 

[Slight]

Rather than the normal probability distribution (with its annoying little left tail), perhaps the data is better modeled by a chi-square [http://en.wikipedia.org/wiki/Chi-square_distribution] distribution?

Hmmm.. I think you would use something like a normal distribution centred around the mean 'skill level' of a population (if such a thing is measurable). You'd use a chi-squared test to evalaute how well test data fits a given hypothesis rather than something like a normal distibution which is handy for modelling population characteristics.

To the OP, I think the problem needs two aspects considered. First is the modelling of the skill level of the population at any particular time (a snapshot). The second is the growth in the actual player base - which may actually resemble the cumulative distribution of a normal distribution. Thus you would expect, for example, at a period of high rate growth, the skill level would be skewed down due to a large number of new players.

There has to be a way to boil down the skill of a player into a number, deaths divided by kills?

In terms of measuring player skill, you have to take into account how long they've been playing and set a benchmark for an average amount of skill after a given length of time . For example: if someone had a 10 deaths and 1 kill, you'd think 'Newbie, no biggie, he can learn' but for someone with 10000 deaths and 1000 kills you'd think 'There is truly no hope for this guy'. But both ways they have a 10:1 Death-to-kills ratio which would give a misleading comparison.

For an example, for something like CoD 4 I'd suggest looking at a Death-Kill ratio alongside the player rank and the % of other Challenges complete.

I do a lot of modelling in my day-to-day work (I help judge markets & strategies for new businesses) so i'd be happy to talk more in a PM - It all sounds quite interesting.

 

[HuCast]

I cant even define skill as long as there are players using glitches or rapid fire controllers on X Box Live maps and stuff like aimbots and wallhacks in counterstrike.Kill/death ratio? Just kill a guy and hide the rest of the game to increase it

 

[rossatdi]

Averted in Left 4 Dead. As long as you can aim your gun to same degree, teamwork and wits are far more powerful than knowing the level/mad skillz. Of course it helps if someone knows which direction.

Shit, "averted"? Must stop reading TV Tropes.

 

[hypothetical fact]

Yeah it's true but I find that the skill level is closer to three weeks in Australia. But that is because it is so had to find other player that don't lag, so when you find lagless games everyone adds each other to their friends lists.

 

[Sion_Barzahd]

A good way to measure a players skill is total played game time, besides K/D ratio.
I don't think matches won can be an accurate way to measure a players skill, as was previously mentioned, you cannot help if you get teamed up with a bunch of retards.

 

[tenny20ca]

I was a PC FPS'er for years then this thing called a Xbox 360 came out. I figured for the cost of a new video card I'd try it out. Did I ever get my @ss handed to me. Once I got use to the controller and the maps though the tables turned. I think the problem here is that people expect to be the best even though alot of people have alot more time to devote to a game. I'd say I'm average in CoD4 and TF2. I have a life, friends, job and other games that eat up alot of my time. If someone wants to spend all thier time playing one game that's fine, I don't expect to beat them. I have a friend that is in the top 30 players for Halo, he plays 40+ hours a week, he eats me alive when we play. I don't get mad or blame the game, he should win, he is ALOT better then me at halo. I've been in some games in CoD4 where I was the top for kills, other games I was the bottom. When I was on the bottom I didn't blame the game, I was just in a game with people that have played ALOT more then me. I don't think you could make a game where experience counted for nothing, unless you make the game totally random, who wants to play lottery shooter 2009? Your gun can fire bullets or healing or blow up in your face, at random. Sounds fun.