Ex-Xbox Boss Reflects on "RRoD", How it Almost Killed Xbox

[Steven Bogos]

Ex-Xbox Boss Reflects on "RRoD", How it Almost Killed Xbox

Ex-Xbox Boss Peter Moore says that the Xbox 360's "Red Ring of Death" cost the company over a billion dollars.

The IGN [http://www.escapistmagazine.com/news/view/86140-A-Detailed-History-Of-The-Red-Ring-Of-Death], explaining that it cost the company over a billion dollars and almost killed the Xbox brand entirely.

"If we don't [address this problem], this brand is dead," Moore told Steve Balmer, the then-head of Microsoft. "It was sickening. I was doing a lot of interviews. It's not like today with social media, which would have been horrific. We couldn't figure it out." Moore says Microsoft initially thought the problem was due to a heating issue, and changed the solders in newer consoles, but was horrified to learn that it still wasn't doing the trick.

"I remember going to Robbie Bach, my boss, and saying, I think we could have a billion dollar problem here," Moore recalls. "As we started to do the analysis of what was going on, we were getting the defectors in, it was a challenging problem for our engineers, and we couldn't quite figure out what it was. We knew it was heat related. There were all kinds of fixes. I remember people putting wet towels around the box."

Moore talks of his nail-biting meeting with Balmer, where he and Bach gave him the cost estimation of their solution - which involved taking back every defective console, fixing them, and FedEx-ing them back with overnight priority mail.

"He said, 'what's it going to cost?' I remember taking a deep breath, looking at Robbie, and saying, 'we think it's $1.15bn, Steve.' He said, 'do it.' There was no hesitation."

Moore said that if Balmer had balked at the price, the Xbox brand would be in a very different place today, and the Xbox One would probably never have been made.

Source: IGN [http://uk.ign.com/articles/2015/07/02/former-xbox-boss-opens-up-about-xbox-360-red-ring-of-death-debacle]

Permalink

 

[RicoADF]

Smart move by Balmer, if Microsoft had of not gone out of their way to deal with RROD then the Xbox and even Microsoft brand would have been suffered far more perminant damage and the fallout would have been worse than it was.

 

[-Dragmire-]

It's not too often that I hear of positive things associated with Balmer.

 

[FoolKiller]

Go big or go home. He went big. It worked.

 

[Ken_J]

Smart move by Balmer, if Microsoft had of not gone out of their way to deal with RROD then the Xbox and even Microsoft brand would have been suffered far more perminant damage and the fallout would have been worse than it was.

Think about were the Game Industry would be if that happened. With Xbox in the toilet a lot of people would have had to deal with the PS3's cell processing bullshit, PC Gaming's dead market or trying to get things to work on the Wii. A lot of people would be out of the industry.

 

[Redlin5_v1legacy]

And their competitors forever hated their far-sighted spending to fix the problem.

 

[The Rogue Wolf]

Did they ever sort out exactly what was causing all the failures? I mean, Microsoft isn't a neophyte in the hardware sector, but to see so many units have problems speaks of a serious design flaw.

 

[EndlessSporadic]

"He said, 'what's it going to cost?' I remember taking a deep breath, looking at Robbie, and saying, 'we think it's $1.15bn, Steve.' He said, 'do it.' There was no hesitation."

It's rare that I read something and stop to think "Holy shit!" in a positive light. This was one of those moments, especially considering who it came from. More importantly, I am completely overcome by the fact that someone acknowledged a mistake they made and wanted to fix it despite the costs.

Of course this all happened after a bunch of denial like everything else when it comes to this sort of thing, but in the end this was still a great move that paid off.

Did they ever sort out exactly what was causing all the failures? I mean, Microsoft isn't a neophyte in the hardware sector, but to see so many units have problems speaks of a serious design flaw.

If i remember right, it had to do with the mounting solution they used to mount the cooler to the motherboard.
When the heatsink got really hot, it caused it to start bending. And because of the way the cooler was mounted, that also caused the motherboard to bend. Which in turn broke solder joints.

Their solution was a new spring mounting system which allowed for more flexing.

Apparently they used a lead-free solder which caused hairline cracks. The multiple instances of heating and cooling only made it more brittle and ultimately caused many units to fail. As they said in the article they tried changing solders and it still failed, but ultimately it came down to poor heat dissipation and the wrong type of solder.

 

[SecondPrize]

The brand is dead in my world. In the end, I had to replace three 360s to continue playing my game collection for it. Never again, Microsoft, never again.

 

[Ryan Minns]

Knowingly releasing a faulty system SHOULD cost them money. They almost got beaten though in "Biggest fuck you" to consumers by a company later on but I think it's probably even.

 

[Imperioratorex Caprae]

Did they ever sort out exactly what was causing all the failures? I mean, Microsoft isn't a neophyte in the hardware sector, but to see so many units have problems speaks of a serious design flaw.

If i remember right, it had to do with the mounting solution they used to mount the cooler to the motherboard.
When the heatsink got really hot, it caused it to start bending. And because of the way the cooler was mounted, that also caused the motherboard to bend. Which in turn broke solder joints.

Their solution was a new spring mounting system which allowed for more flexing.

Apparently they used a lead-free solder which caused hairline cracks. The multiple instances of heating and cooling only made it more brittle and ultimately caused many units to fail. As they said in the article they tried changing solders and it still failed, but ultimately it came down to poor heat dissipation and the wrong type of solder.

It was a bit of both. The original design called for a more flexible x-clamp but the company that manufactured the public consumption 360's went with a more rigid clamp. That paired with the heat dissipation issues and the solder choice made for a 3-stage failure. It may have been cost-cutting by Microsoft directly or the manufacturer just pocketing more cash for less expensive materials, the situation is murky at best but having Ballmer directly take action after being advised is a cool thing. Also MS did repair most Xbox's that failed for free, including sending a postage paid box. My wife and I owned an original xbox that failed for other reasons and Microsoft still sent us a free repair, so they gained a huge plus in my book.
In my view, companies can screw up but its how they handle said screw ups is what makes or breaks them. Microsoft screwed up, whether it was a cost-cutting gaff or the manufacturer's fault I don't have the information from reliable sources to say, but they fixed it for free and didn't require people to buy a new system. They also changed the design of newer releases to have better heat dissipation and flexible clamps. We had an Elite (the black box) and it never failed in the 4+ years we had it, and that thing got major usage.

 

[Amaror]

Well it's not like it was a first really. I still remember some the original xbox there were some editions who had reported fires breaking out in the box, when it was constantly connected to power.
Then they just send a better powercable that could be turned off and on so the box would only get power when it needed it.
That also worked, my old xbox is still running without problems. Though i still haven't figured out how they knew i had an xbox. They didn't even adress the package with the new powercable to my mother who bought it, but directly to me.

 

[J Tyran]

As annoying as it was to have my launch bought Xbox 360 die within 6 months the way Microsoft handled did take the sting out of out, it happened at the weekend and I called them. Monday a courier bought a box, came back on the Tuesday to collect the stripped down (HHD taken out etc) and packaged XBox away and then on Thursday dropped another new/refurbished Xbox 360 off.

A fairly fast and painless process as these things go, so I will give them that at least.

 

[CrystalShadow]

Just shows you how bad it was. No console I've ever owned has failed, and some of them are close to 20 years old.
Some have a tiny bit become glitchy, (usually more so individual games though), but none have ever failed.

A PC I had did fail. It came from a period where a very large batch of capacitors being used for many motherboards were faulty.

Also had a few bits of cheap chinese 'off brand' devices fail on me... But that's different in general.

 

[Darks63]

This must have happened after I dropped the Xbox brand. I had 3 consoles RRoD on me in the span of 6 months and needless to say I converted back the PS brand and didn't look back.

 

[Evonisia]

Given the 360's longevity and the One's success, the move payed off even if they didn't make the money back. I still remember my friend's 360 RRoD'ing which I think was very shortly after I got mine, which was a scary sight to say the least.

 

[Frezzato]

Just shows you how bad it was. No console I've ever owned has failed, and some of them are close to 20 years old.
Some have a tiny bit become glitchy, (usually more so individual games though), but none have ever failed.

A PC I had did fail. It came from a period where a very large batch of capacitors being used for many motherboards were faulty.

Also had a few bits of cheap chinese 'off brand' devices fail on me... But that's different in general.

Just curious about the PC with the faulty capacitors. Was it a Dell?

 

[CrystalShadow]

Just shows you how bad it was. No console I've ever owned has failed, and some of them are close to 20 years old.
Some have a tiny bit become glitchy, (usually more so individual games though), but none have ever failed.

A PC I had did fail. It came from a period where a very large batch of capacitors being used for many motherboards were faulty.

Also had a few bits of cheap chinese 'off brand' devices fail on me... But that's different in general.

Just curious about the PC with the faulty capacitors. Was it a Dell?

Nah. Custom build. Lost 2 different mainboards that day within about 2 years. Can't remember what manufacturer it was.
I know the first system was an AMD duron system, while the second was a Celeron from the era when slot 1 pentium 3 systems were a thing...

Fairly sure the Mainboards were different brands too.
This was a rather widespread issue. Lots of bad power capacitors.

Adverising these days pointing out the use of 'high quality solid state power capacitors' (or something similar) can probably be traced to the widespread problems back then.