343 Apologizes as Halo: MCC Continues to Be Troubled

[Callate]

You get to play the game ahead of time after all major bugs had been fixed. While you're saying "I'm an unpaid beta tester!"... no, you're not. You're playing a game in a beta state. You are not shuffled into a tiny office with a dozen other people and forced to replay the same area over and over and over again doing the exact same thing trying to find insignificant bugs, then fill out a several-page report documenting each glitch you find. While not being able to say a damn thing about it because of an NDA.

Instead, you get early access to a game that may have a few technical flaws, but you can play at your own pace and leisure, and possibly talk freely about with your friends, doing your own thing, and maybe sending a brief ticket if you come across something game-breaking, if you feel like it.

For buying something you intended to buy anyway, while having a service you already use, and enjoy a popular series, you can get early access to the next game in the series. That is a very good deal.

Beta-testing is not defined by being in a controlled corporate environment, or signing a Non-Disclosure Agreement, or repeating a particular segment of game repeatedly in order to weed out specific bugs. Beta-testing is defined by testing software that is in a beta state. If you telecommute to test code that is in a beta state, you're still beta testing. If you're playing a beta game with the goal of getting a wide view of the overall experience and the developers are getting data back from your play, you're still beta testing. If it's three weeks to launch and the developers don't require an NDA of the people playing beta code because getting network traffic information from a sufficiently large sample is more important that a few leaks, you're still beta testing.

Now: is the work done by someone who's paid $8.25 an hour or so to sit in a small space and try to figure out why a quest flag isn't tripping by repeatedly re-playing the world's most boring fetch-quest more onerous than the work done by someone who got a key in a box for some other product and has the option of ignoring it, or wigging out any time?

Yes.

But the latter is still beta testing- and, in fact, the mass-consumer pitch serves to make a lot of that low-wage, small-cubicle testing unnecessary; arguably, may put some of those same people out of work.

Further, the beta test included in TMCC is extremely limited- only four player, and only for two weeks, and only for those who are paying for Gold membership. Not exactly "doing your own thing". Some players may enjoy it, sure. But "a very good deal"? It's still unpaid beta testing work wrapped in the mantle of a "freebie" in a consumer offering that is very much not beta, yet still clearly suffering from its own share of issues.

So if you simply want to make the case that "professional" beta-testers have the more difficult lot, well, sure, I agree.

But what the non-professionals are doing is still unpaid beta-testing, and arguably even beta-testing they're actually paying the company for the privilege of performing on their own equipment, in their own premises, and on their own time. "A very good deal"? For the company, yes. For the consumer? I think we'll have to agree to disagree on that point.

 

[Scow2]

But what the non-professionals are doing is still unpaid beta-testing, and arguably even beta-testing they're actually paying the company for the privilege of performing on their own equipment, in their own premises, and on their own time. "A very good deal"? For the company, yes. For the consumer? I think we'll have to agree to disagree on that point.

The beta-testing they're doing is not the same as the beta-testing beta-testers are hired for. Calling these guys "beta testers" is like referring to people who go to "Pick your own fruit!" farms as "Farmers".

They get all the perks of beta testing (Trying a new game before everyone else does!) after all the 'real' beta testers have already done their jobs, without any of the indignities of beta testing.

It's not even comparable to "Telecommuting" - While they have limited access to just 4-player multiplayer, they are not required to perform specific activities, are not required to fill out bug reports, nor anything else beta testers are assigned to do. And, they don't threaten real beta-testers jobs, who still have to get the game in a playable state before the promotional beta.

It's a win-win situation, and it IS good for the consumer.

 

[Callate]

The beta-testing they're doing is not the same as the beta-testing beta-testers are hired for. Calling these guys "beta testers" is like referring to people who go to "Pick your own fruit!" farms as "Farmers".

They wouldn't be farmers, they would be pickers. (Much like the professional beta testers aren't game producers or designers.) And if after the "u-pick" crop season was done the farmer went on to hire pickers to finish emptying the fields, or to harvest other fields set aside for commercial use, those pickers would also be doing a onerous, tedious, and under-paying job- but one with an unavoidable similarity in its basic mechanics to the people who came to pick their own fruit.

They get all the perks of beta testing (Trying a new game before everyone else does!) after all the 'real' beta testers have already done their jobs, without any of the indignities of beta testing.

It's not even comparable to "Telecommuting" - While they have limited access to just 4-player multiplayer, they are not required to perform specific activities, are not required to fill out bug reports, nor anything else beta testers are assigned to do. And, they don't threaten real beta-testers jobs, who still have to get the game in a playable state before the promotional beta.

It's a win-win situation, and it IS good for the consumer.

Firstly, in an era where it's infamously common for "finished" commercial software to contain significant bugs and errors and in some cases require lengthy patch downloads, some of the indignities of beta testing seem to have been spread very widely indeed.

It's also more than likely that at least some errors, bugs, network snafus etc. are going to be present in the multiplayer beta that will not be in the final product; were this not the case, there wouldn't be a need to pull a large sample of random testers in the first place.

While one would hope that Microsoft and 343 employ a higher standard than, say, the various "Early Access" options at Steam, they're still offering a product that they don't believe is "ready for prime time".

Secondly, it's partly the matter of scale that allows the consumer "key-testers" to do some of the work that might otherwise be done in a professional testing lab environment. If one is trying to find a needle in a haystack, five people who have a pretty good idea where the needle fell have a decent chance, but so do a hundred people who only know it's somewhere in the haystack. If those five are hired to do the job and required to stick to it and those hundred are volunteers and a dozen or more may wander off before the job is done, the job still gets done.

This sort of work was performed before public betas became a relatively common thing. Some varieties still are performed privately, and the balance between private and public testing remains something many companies continue to struggle with- as evidenced by a number of highly visible problems in new networked-game launches.

One of the things a public beta promotes is a sense of ownership in that game's community, and with it, a greater willingness to forgive certain problems and errors that were there when the games were "in conception", because that community felt it was part of that conception. It is difficult not to feel that game players are increasingly being conditioned to accept unfinished work.

I think I understand where you're coming from, believe it or not. But I still see the difference between professional beta testers and unpaid consumer keytesters as largely one of degrees, and one that I suspect companies would eliminate if they could, and some are in the process of doing even now (as, again, examples like Steam's Early Access would seem to attest.) And while companies like Microsoft and 343 have been very successful in selling early beta access as an additional benefit, I remain skeptical that on balance the benefit isn't weighed mostly on the company's side.