Poll: Why not use benchmark tests instead of demos?

[Smooth Operator]

Well it's a tiny step better then a plain sheet of system requirements(because official specs are mostly complete fabrications), but still eons away from what a demo tells you, it isn't about the score at the end you need to know if the game responds in a way that is enjoyable.

 

[zehydra]

I would much prefer benchmark tests to demos myself, but I don't see anyone big doing this anytime soon.

If EA or some such were to announce such a thing, the most common response from the consumer base would be "Uh, what? What is this nerd nonsense?? Just give me something to play! Fucking shit ass homofaggots"

It's a great idea, but Demos are quite popular among those who don't really care about specs

 

[chozo_hybrid]

No. Because there is only one system where Benchmarking is applicable - PC, and games on this system come with two very important things already - Graphics options (Low, Med, High for example) and minimum/recommended hardware requirements.

With those things already I know whether my rig can run it and roughly to what level.

There is also "Can you run it?" (put it into google) which tells me how well my computer can run specific games.

But not only is there already far more efficient, specific ways of finding out the information your "Idea" aims to give us, your idea is just plain stupid from a practical stand point

How many different configurations of computers are there? Different pairings of CPU's, HDD's, GPU's, SSD's, RAM, Monitor resolutions...all with varying performance. How exactly would you benchmark a game? Would you write up pages and pages of graphs for each possible configuration? Sounds real fuckin' neato.


To sum up - The information is already accessible in a less dumb way. Thanks for playing, have a nice day now.

True, the poll I made was just to see what people woould think if they replaced demos. But considering how few games release demos, I think benchmark tests would at least be something to show it's playable on your machine. Because I've been within recommended (Note that I didn't say minimum) specs for a few games and they ran like crap unless I put everything on minimum, which ruins it a bit.

I'm not a graphics mean everything kind of guy, but when I buy a game, I like it to look how the devs intended it too.

And how is it then, that some games come with them? They don't have to write up pages of graphs etc, it literally just gets you computer to run through a section of the game and graphs the frame rate, giving you the high points, low, and the over all average.

Flat out saying it's pointless is kinda silly, when you consider that it would yet be a another option, and something more concrete then just a few numbers and words telling you what a game needs. Especially when they don't give out enough information, or can be kinda wrong as they have been in the past with the odd game. As someone earlier in the thread pointed out, Dishonored played fine on his computer despite some of his specs being lower then the required. Those same specs stopped me from buying the game, because I thought it wouldn't run well. With no demo out for the game, a test would at least have shown me it would work okay on my machine, what's wrong with that.

I agree, I have thought of this myself for a long time. Personally I would prefer this over a regular demo since it gives you more detailed information than a demo. It tells you how well your computer can run it, recommended settings, average framerate, framerate during stress. A demo can be deceiving because it usually shows the start of the game and that might not always reflect well on how intensive it can get.

However I voted other because it wont affect my purchases. My computer plays the games that are being released. At worst I have to play on medium which isn't a dealbreaker. However if I still had my old computer I would love this. I hope this will one day be a reality for all games.

Yeah, I mean don't get me wrong, I would prefer a demo over a benchmark test, but so few demos tend to get released that this seems like it would be a good idea.

 

[New Frontiersman]

Because benchmark tests don't tell you what the game is like or if it's fun to play.

What if I see a game that looks interesting but I've never played one of that type before? A demo would help me tell if it's the kind of game I would enjoy. Plus a benchmark test wouldn't reveal things like glitches or broken gameplay which might be revealed in a demo.

Plus people who might want to play a game on console have no use for benchmark tests whatsoever, every game made to run on a console is designed to run well on that console by default unless the game is absolutely broken, so a benchmark test likely won't tell you anything you wouldn't already know.

Not that I'm saying that benchmark tests are a bad idea, just that there are other uses for a demo besides telling you if a game works on your computer, so benchmark tests can't entirely supplant demos.

 

[BrotherRool]

Besides- demos used to be a standard part of game marketing. Devs should stop slacking off and give us the same pre-launch treatment they all USED to do.

Unfortunately there's a very good reason people don't use demos for marketing often anymore. It loses sales almost always.
http://www.penny-arcade.com/patv/episode/demo-daze

Basically it's hard to make a demo look good and if the demo is good it there's a portion of people who play the demo and don't buy the game

 

[DazZ.]

How exactly would you benchmark a game? Would you write up pages and pages of graphs for each possible configuration? Sounds real fuckin' neato.

You'd run a benchmark test... which is exactly what this persons suggesting. Where it tells you exactly how your specification would handle the game as it gets the framerate average from your setup...

Why do you think you'd need graphs? You know what a benchmark test is yes?

 

[Yopaz]

I agree, I have thought of this myself for a long time. Personally I would prefer this over a regular demo since it gives you more detailed information than a demo. It tells you how well your computer can run it, recommended settings, average framerate, framerate during stress. A demo can be deceiving because it usually shows the start of the game and that might not always reflect well on how intensive it can get.

However I voted other because it wont affect my purchases. My computer plays the games that are being released. At worst I have to play on medium which isn't a dealbreaker. However if I still had my old computer I would love this. I hope this will one day be a reality for all games.

Yeah, I mean don't get me wrong, I would prefer a demo over a benchmark test, but so few demos tend to get released that this seems like it would be a good idea.

Personally I think demos can be deceiving both in terms of capabilities and in quality. I remember playing the demo for Bioshock, I found that I liked it and that it was running perfectly. I decided to buy it. It turns out that I didn't really like the game and also as soon as I met a turret the game slowed down so much it was unplayable.

If I had run it through a benchmark I would have known the upper and lower limit of the framerate and just as much about the quality of the game as I did after playing the demo. Demos are more fun, but benchmarks are more useful, I guess I am more of a practical person in that regard.

 

[veloper]

So you'll know if the game will run with plenty frames per second, but you don't learn anything about the gameplay being fun or not?

I've always been able to run every game I ever wanted on my rig. A benchmark demo is not useful to me.

 

[chozo_hybrid]

Snip

All valid points, I did say in my post I would rather see both or at least a demo come out, but they are so few and far between that these tests at least would give us something. As for consoles, yes, they would be useless, but I didn't factor them in because there was no point, I was referring more to PC games.

Snip

I get that some wouldn't use it so people don't know, but I imagine you wouldn't really decrease sales on games by allowing benchmarks to become available, since if someone was on the fence and weren't sure their computer could handle it, most probably wouldn't get it until they upgrade or get something else instead.

I agree, I have thought of this myself for a long time. Personally I would prefer this over a regular demo since it gives you more detailed information than a demo. It tells you how well your computer can run it, recommended settings, average framerate, framerate during stress. A demo can be deceiving because it usually shows the start of the game and that might not always reflect well on how intensive it can get.

However I voted other because it wont affect my purchases. My computer plays the games that are being released. At worst I have to play on medium which isn't a dealbreaker. However if I still had my old computer I would love this. I hope this will one day be a reality for all games.

Yeah, I mean don't get me wrong, I would prefer a demo over a benchmark test, but so few demos tend to get released that this seems like it would be a good idea.

Personally I think demos can be deceiving both in terms of capabilities and in quality. I remember playing the demo for Bioshock, I found that I liked it and that it was running perfectly. I decided to buy it. It turns out that I didn't really like the game and also as soon as I met a turret the game slowed down so much it was unplayable.

If I had run it through a benchmark I would have known the upper and lower limit of the framerate and just as much about the quality of the game as I did after playing the demo. Demos are more fun, but benchmarks are more useful, I guess I am more of a practical person in that regard.

That is exactly what I am getting at, because some demos are made with certain things cut out etc, it can be a deceptive representation of what can run the game, where as these tests as far as I know just show the fact of how well your PC will run at the settings you choose for the test. I got Metro 2033 as part of the THQ bundle and went to play the game and it went slow, after benchmarking I was able to work out what I needed to change to make it playable.

So you'll know if the game will run with plenty frames per second, but you don't learn anything about the gameplay being fun or not?

I've always been able to run every game I ever wanted on my rig. A benchmark demo is not useful to me.

So because it's not useful to you, that makes it a useless thing for devs to do? I understand where you're coming from, but not all of us have the luxury of building a really power gaming PC.

Considering how barely any demos are released these day, what harm would a benchmark test do, at least then you can be sure or not your computer would run it.

 

[Yopaz]

No. Because there is only one system where Benchmarking is applicable - PC, and games on this system come with two very important things already - Graphics options (Low, Med, High for example) and minimum/recommended hardware requirements.

With those things already I know whether my rig can run it and roughly to what level.

There is also "Can you run it?" (put it into google) which tells me how well my computer can run specific games.

But not only is there already far more efficient, specific ways of finding out the information your "Idea" aims to give us, your idea is just plain stupid from a practical stand point

How many different configurations of computers are there? Different pairings of CPU's, HDD's, GPU's, SSD's, RAM, Monitor resolutions...all with varying performance. How exactly would you benchmark a game? Would you write up pages and pages of graphs for each possible configuration? Sounds real fuckin' neato.


To sum up - The information is already accessible in a less dumb way. Thanks for playing, have a nice day now.

I just ran a benchmark test on Total War Shogun 2 on my computer. It told me that I run the game in 1080p with DX11 with the average of 38 fps where the lowest was 32 and highest 54. It used the game's engine and tested how it would run with my specs. It took me about 5 minutes and I got a message telling me the average framerate and a log I could check to see more detailed information. How is this dumb?

 

[NSGrendel]

Snip

All valid points, I did say in my post I would rather see both or at least a demo come out, but they are so few and far between that these tests at least would give us something. As for consoles, yes, they would be useless, but I didn't factor them in because there was no point, I was referring more to PC games.

Snip

I get that some wouldn't use it so people don't know, but I imagine you wouldn't really decrease sales on games by allowing benchmarks to become available, since if someone was on the fence and weren't sure their computer could handle it, most probably wouldn't get it until they upgrade or get something else instead.

You imagine wrong. You are assuming that people make logical decisions when purchasing. They do not. I wish they did, since it would make the world a much less annoying place.

Besides which - you do realise what would happen if this actually became a standard? Yup, that's right. Optimised benchmarks for games that run like shit when fully released.

And if you retroactively applied this to gaming - many excellent games perform like crap when released. If we applied a benchmarking test to Deus Ex or Shogun: Total War when they came out, they would have sold a dozen copies between them, due to the fact that Deus Ex could barely function running DirectX and Shogun took eons to load.

 

[NSGrendel]

No. Because there is only one system where Benchmarking is applicable - PC, and games on this system come with two very important things already - Graphics options (Low, Med, High for example) and minimum/recommended hardware requirements.

With those things already I know whether my rig can run it and roughly to what level.

There is also "Can you run it?" (put it into google) which tells me how well my computer can run specific games.

But not only is there already far more efficient, specific ways of finding out the information your "Idea" aims to give us, your idea is just plain stupid from a practical stand point

How many different configurations of computers are there? Different pairings of CPU's, HDD's, GPU's, SSD's, RAM, Monitor resolutions...all with varying performance. How exactly would you benchmark a game? Would you write up pages and pages of graphs for each possible configuration? Sounds real fuckin' neato.


To sum up - The information is already accessible in a less dumb way. Thanks for playing, have a nice day now.

I just ran a benchmark test on Total War Shogun 2 on my computer. It told me that I run the game in 1080p with DX11 with the average of 38 fps where the lowest was 32 and highest 54. It used the game's engine and tested how it would run with my specs. It took me about 5 minutes and I got a message telling me the average framerate and a log I could check to see more detailed information. How is this dumb?

Well, the above is immediately misleading, because despite how much I love Shogun 2, it takes over a minute to load each battle whilst playing. Frame rate is completely worthless in measuring a game like Shogun 2, since decision making occurs over many seconds. On the other hand, since you fight a LOT of battles if playing the game without battle "autocomplete", an extra 1 to 2 minutes of loading every 2 minutes of gameplay on the campaign map is a LOT of sub-optimal gameplay.

 

[chozo_hybrid]

You imagine wrong. You are assuming that people make logical decisions when purchasing. They do not. I wish they did, since it would make the world a much less annoying place.

Besides which - you do realise what would happen if this actually became a standard? Yup, that's right. Optimised benchmarks for games that run like shit when fully released.

And if you retroactively applied this to gaming - many excellent games perform like crap when released. If we applied a benchmarking test to Deus Ex or Shogun: Total War when they came out, they would have sold a dozen copies between them, due to the fact that Deus Ex could barely function running DirectX and Shogun took eons to load.

I imagine wrong; you know this for a certainty? Yes there are some dumb people out there, but you can't base that assumption on everyone that's just pessimistic.

Of course you'd probably see people using it to deceive people, much like some demos have done in the past, but then most would quickly learn not to trust that particular company like people do with all sorts of things and not buy anything from them.

But think of the potential for good this could have on the industry as well, you have to at least acknowledge that too, they both sides of the same coin.

 

[Yopaz]

I just ran a benchmark test on Total War Shogun 2 on my computer. It told me that I run the game in 1080p with DX11 with the average of 38 fps where the lowest was 32 and highest 54. It used the game's engine and tested how it would run with my specs. It took me about 5 minutes and I got a message telling me the average framerate and a log I could check to see more detailed information. How is this dumb?

Well, the above is immediately misleading, because despite how much I love Shogun 2, it takes over a minute to load each battle whilst playing. Frame rate is completely worthless in measuring a game like Shogun 2, since decision making occurs over many seconds. On the other hand, since you fight a LOT of battles if playing the game without battle "autocomplete", an extra 1 to 2 minutes of loading every 2 minutes of gameplay on the campaign map is a LOT of sub-optimal gameplay.

Sure, it might not be perfect for a game like this, but this was an example of how it can be done, how simple it is and as an example of what a benchmark can tell you. The guy I quoted seemed to think this was a complex process rather than running a program for 5 minutes.

I would also like to disagree with your statement that framerate is completely useless. I have played the game with an average of 20 fps and it was not pleasant. With this I know that while I'm playing it's quite smooth as opposed to the choppy 20 fps I had once before.

 

[NSGrendel]

You imagine wrong. You are assuming that people make logical decisions when purchasing. They do not. I wish they did, since it would make the world a much less annoying place.

Besides which - you do realise what would happen if this actually became a standard? Yup, that's right. Optimised benchmarks for games that run like shit when fully released.

And if you retroactively applied this to gaming - many excellent games perform like crap when released. If we applied a benchmarking test to Deus Ex or Shogun: Total War when they came out, they would have sold a dozen copies between them, due to the fact that Deus Ex could barely function running DirectX and Shogun took eons to load.

I imagine wrong; you know this for a certainty? Yes there are some dumb people out there, but you can't base that assumption on everyone that's just pessimistic.

Of course you'd probably see people using it to deceive people, much like some demos have done in the past, but then most would quickly learn not to trust that particular company like people do with all sorts of things and not buy anything from them.

But think of the potential for good this could have on the industry as well, you have to at least acknowledge that too, they both sides of the same coin.

Yes. I know this for as close to a certainty as I can. It has been sadly drummed into me by personal experience in my career, but feel free to use Google to verify by looking at studies on buying behaviour, conversion motivation or just simply "+emotion +human +decision +making".

Bottom line - selling stuff is about making people feel good. People with good PCs already feel good. By giving them a mechanism to potentially feel bad, you are only reducing your ability to sell. Don't take my word for it. Read up/around the subject in hand. Just be prepared to let go of all your previous beliefs, because unless you've had a very harsh upbringing, it will invariably undermine your faith in humanity.

Do you doubt that the majority of people refuse to believe demonstrable facts over their own feelings? Because that's what you're suggesting.

 

[chozo_hybrid]

Snip

What career might that be? I'm curious, since you're using it to back you up. I work in retail and yes, to a degree I know what you're talking about, but that doesn't apply to everyone. But that's just a vocal idiotic few in my experience.

Why would it make them feel bad if they can't run a game, it would feel a lot worse buying one and finding it doesn't work for you. It's a whole lot of money potentially down the tubes, unless you're willing to spend more to get it running.

Bottom line, actually, is that the companies want to make money, and if they were smart, they wouldn't sabotage themselves with a benchmark that doesn't actually do its job at informing the consumer how well it will run. People figure out when something like that is done, then they spread the word to others, that company loses the trust of people and it could damage them a lot.

I looked at what you suggested, and yes, I see your point. People usually go with their feelings on something, but I know that with me, after being bummed about seeing through a benchmark a game won't run on my machine, I'll be glad I didn't purchase it day one and get even more disappointed by finding out something I forked out $100 for didn't work on my machine, despite specs saying I should be okay. Which has happened to me once or twice, and I've learned from.

 

[2xDouble]

Can someone explain what a benchmark test is? Unfortunately i have no clue .

It's a little app that detects your computer's resources and measures that against what the game needs. It's something people run because they can't or don't want to read tech specs.

... Or the tech specs are horribly inaccurate like Dishonored's. It's listed minimum requirement is a GTX 460, but the game can be run smoothly on an 8800GT.

Right, that too. heh.

 

[ResonanceSD]

Can someone explain what a benchmark test is? Unfortunately i have no clue .

It's a little app that detects your computer's resources and measures that against what the game needs. It's something people run because they can't or don't want to read tech specs.

Not even slightly. A benchmark test is a program within the game which loads up textures, polygons, and scenes from games and stress tests your hardware to see how good it is against the game's engine.

Can someone explain what a benchmark test is? Unfortunately i have no clue .

Please don't listen to this guy.

 

[josemlopes]

Yes, I think benchmarks help inform a decision to buy a game, so yes for the poll.

Would I want benchmarks to replace demos and think it's a good idea?

HELL NO!!!

I dont think he meant that, I think he just meant that some developers dont make demos because its hard (at least time consuming) to implement one in the game and sometimes some games can look worse worse because of a demo (Dragons Dogma looks like a weak game because of the demo limitations) so they should use benchmark tests for these cases.

Think of how would you make a demo of Skyrim, you could limit a small area or just give 3 quests to complete but if an unninformed player plays that he wont get the right idea of what the game is actually about and its scope.

Spec Ops The Line has a demo that has nothing to do with what makes the game worth a buy. From the demo you only get a generic third person shooter.