Do Game Review Scores Matter? 88
jasoncart writes "This piece on Ferrago discusses the use of review scores and the significance they play in gamers purchasing decisions. Ultimately, according to columnist Ben Parfitt, review scores are pointless." From the article: "Few things wind me up more than when what appeared to be a well-balanced and thoughtful gaming discussion descends into a successive barrage of review scores and Gamerankings ratings."
Take them with a grain of salt (Score:4, Informative)
Re:Take them with a grain of salt (Score:1)
Re:Take them with a grain of salt (Score:2, Interesting)
Reviewers are robots. (Score:5, Insightful)
The best judgement of a game (because of the above), is to see anomalies in gameranking.com listings. If IGN or Gamespot or Gamespy or EGM give a review that seems to be an anomaly, ignore it. It's a bought review.
Always read the review giving a game the lowest score. Or stop reading reviews with scores altogether. The latter works well for me.
Re:Reviewers are robots. (Score:2, Interesting)
Read the reviews of the game but in the end don't read the scores they are given, because if the game doesn't require a computer faster than has ever been built to render it will not display well.
Re:Reviewers are robots. (Score:2, Funny)
Moral to this story: don't post from work!!!
Re:Reviewers are robots. (Score:5, Interesting)
You can be sure that at Gamers.com [gamers.com] there are no bought reviews. In fact, in certain situations we've even received hate email by developers and publishers for not praising their game. We report what we consider to be fair and just, despite all the buzz.
Of course, not everyone should trust a single reviewer's opinion over a particular title. That's why we have a special section (Newest Games [gamers.com]) in our forums to let new games fall under even more scrutiny by our members.
Between reading a review and following up on other gamers opinions, one should have a rough idea if the game is worth purchasing.
Re:Reviewers are robots. (Score:3, Interesting)
The upside to that is if you don't get angry when a game is poorly-received, it means that you weren't passionate enough about making it.
Re:Reviewers are robots. (Score:4, Insightful)
Gamers might not get money for review scores, but you guys seem to have an uncanny ability to give scores thatseem to match the text of the reviews. For example:
And, right after claiming it's one of the best games of the year, the score is 3/5. The same score that "Hamtaro ham-ham games" and "The punisher", where the text of the review says "but to spend $49.99 on this game only proves that you're either a huge Punisher fan (which is fine) or that you enjoy "punishing" yourself". To me, that makes as much sense as Chewbacca living in Endor.
Bigger sites coomit the same sin (just read the Halo 2 review in gamespot, and then look at the score), but if you guys want to gain the audience that gamers once had, you have to do better than this.
Re:Reviewers are robots. (Score:3, Interesting)
And you have to stop halting my experience with full screen ads every page because I refuse to let you put a tracking cookie on my computer. The "Click here to skip this ad" is also barely visible in Firefox.
I know you need to feed the bandwidth family, but please come up with a less obtrusive advertisin
Re:Reviewers are robots. (Score:2)
This type of ad is called a rich media ad. You should only see it once every 100 times you visit Gamers.com, so we didn't think it was that much of an intrusion. Personally, I've never seen the ad (and I go to the site 10 times a day).
However, the entire reason we exist is for people like yourself, so I'm going to forward this up the chain and see if we can come up with a better solution.
Re:Reviewers are robots. (Score:2)
The rich media ad could be dumped easily by moving the advertisment below the fold on the left to the right above the fold. Stretch out the top banners, make the page your typical 3 column jobber, and boom, you've got more advertising space above the fold and it's relatively unintrusive. Of you can go the route we went at f13 and make everything modular.
On another note, if you want to convince people that your ads are worth r
Re:Reviewers are robots. (Score:2)
You have this backwards. Anomalies are usually the independent reviews, and will more often than not give you closer to a "true" score for a game (if there is such a thing - games are pretty subjective, moreso than a lot of other "arts").
Actually, based on my five years in the game industry, f
article summary (Score:5, Insightful)
a system meant to reduce all that is to be said about a game to a single-digit number somehow leaves the smaller nuances unsaid.
stunning.
Huh? (Score:5, Funny)
Well, duh. (Score:1, Interesting)
For game review figures I mostly only care about very rough breakdowns... THe five-stars system is good that way because it gives you a rough idea of "Terrible" "Poor" "Okay" "Good" "Great" without getting all hung up on whether game x is
Answer: NO! (Score:3, Interesting)
Re:Answer: NO! (Score:3, Insightful)
Example?
Not challenging ya really, I was just wondering if a AAA game company released a 1.0 stinker.
Re:Answer: NO! (Score:2)
Any of the Matrix games from Atari/Infogrames...
The latest Tomb Raider (although maybe that deservers 2/10...)
Re:Answer: NO! (Score:2)
Oh, I think I remember one [firingsquad.com]. Worst $4.99 bargain bin buy ever...
(your point still stands)
Re:Answer: NO! (Score:2)
At least it competes for title of Worst 4.99 bargain bin buy ever.
$4.99? I can top that. (Score:3, Interesting)
http://www.netjak.com/review.php/537
Re:$4.99? I can top that. (Score:2)
We combine stong management ( with our Santa Monica, CA based headquarters ) with superb creative, yet non expensive development teams in Russia. This allows our clients to cut development expenses 3-5 times comparing to cost of local European or US development team or employees. At same time we provide instant feedback to our clients, thanks to our management offices in Los Angeles and London.
And, apparently, their writing is outsourced to Bangladesh. But hey
Q:Do Game Review Scores Matter? (Score:3, Insightful)
Interesting, Mod Parent up (Score:2)
Scores help to narrow which games to look at. (Score:3, Insightful)
Independent reviews are better (Score:4, Interesting)
Professional reviews are useful for a very general overview of a game, but usually it is easier to find important details at places like GameFaqs. People posting on their own without a profit motive are more likely to mention that a game is really short or overly linear, for example. Of course the noise to signal ratio is very high, but the information is there for people with a little patience.
Re:Independent reviews are better (Score:2)
Re:Independent reviews are better (Score:1)
Re:Independent reviews are better (Score:2)
You are right, but if the indy reviews are read properly, there can be a useful nugget of information that is absent from professional reviews. If a game is particularly buggy, there will be no question of it on the discussion boards, for example.
Re:Independent reviews are better (Score:1)
Do film review scores matter? (Score:2, Interesting)
I personally thought kill bill 1 and 2 were so pretentious, oh, but they were cool movies, and how dare I speak out against them... people seem to think you are challenging them if you do not agree with thier movie viewing habits.
Look, kill bill(s) were shit IMHO. ok, deal with it, love it. move on.
Game reviews on the other hand, or music reviews. If you look at the Lemon Jelly website they are talking about thier mixed reviews, some say it
Re:Do film review scores matter? (Score:1)
Meta sites and knowing the reviewers (Score:5, Interesting)
Sites like game rankings [gamerankings.com] give you the review numbers from a number of sites. Given that you get a fairly good idea of where a game sits. For films movies [movies.com] does the same for films. Given that and active reviewers on the site gives you a fair idea of what is good.
Knowing your reviewers is the other way to get good information. If you regularly read a particular reviewer you'll get a good idea about what they like and what they don't like. This is easier with films than games, but still possible.
Reviews are definitely subjective, but are still a useful way to make your money and time go further and if a bit of thought is used are well worth looking at.
Re:Meta sites and knowing the reviewers (Score:2)
Why bother? (Score:1, Offtopic)
Honestly, why go to that trouble and then distract or drive off a large chunk of your readership?
Wow this guy's wound up (Score:2)
"Does the quality of a game and the sum of the efforts put into its creation merely boil down to nothing more than a single or double digit number?"
No. It's a measurement of somebody's reaction to it. It's not a bad one, either. If a game rates an 8.2, for example, then it means that the reviewer thought it was pretty decent, just not earth shattering. If a reviewer says the graphics are 70%, then he's saying that it's not th
Re:Wow this guy's wound up (Score:2)
No, his problems are with the review scores, and I agree with him. This has also been a debate in the film industry ever since Siskel a
Do Game Review Scores Matter? (Score:2, Insightful)
I use scores as one of the elements... (Score:2)
Creating a meaningful average (Score:4, Interesting)
The article points out the flawed logic inherent in the system of averaging random scores assigned on a 0-10 scale. Is it possible for a game with an 8.1 average rating to be better than a 9.1 average rating? Yes. Everone is entitled to their own opinions. The author cites an example of where in his opinion a game with an 8.1 rating is more enjoyable than a game with a 9.1. Apparently the author was chastised for expressing his opinion. This is a downside to averaging: it can lead to groupthink.
But what can we do to combat groupthink? Consider the following simple ranking systems:
At first glance it appears that any one of these systems would work adequately if used consistenly and then averaged for at least 30 reviewers. The average scores should then in theory be meaningful, right? Well unfortunately we have to note the key words there: 'used consistently'. If the reviewers cannot agree on a format, then you have to reduce it to the lowest common denominator. Similarly, many reviewers would simply ignore the 'recommended' option in favor of the extremes. This suggests that perhaps the best option is to average the binary review score.
But wait! What if the system gets flooded with artificial reviewers? This happened in recent memory when Sony admitted inventing fictitious reviewers to gush about the movie "A Knight's Tale". What if those artificial reviewers get included in the average? That is a serious problem, but it's easily addressed with moderation! Examine each reviewer's track record before adding them to the mix. And then pull any reviewer that is consistently out of touch with reality.
Recommendation: Find a bunch of games you like and a bunch of games you dislike. To be thorough, you want to find at least 30 in each category. Search out critics that agree with your tastes for at least 2/3 of the titles. Average the opinions of these critics when a new release comes out. If the result comes out at least 2/3 (0.67), then you'll probably like the game.
Addendum: For better results, you can assign weights to certain critics and then perform a weighted average. For example, you might observe that critic A agrees with you 90% of the time, while critic B only agrees 80% and critic C agrees 70%. In this example,if only C dislikes the game, then your result will be greater than 2/3 (favorable); however, if A dislikes it, then the result will be less than 2/3 (unfavorable). Keep in mind that to be statistically meaningful, you need to have at least 30 reviewers, and also remember that if you get burned by a critic, you can always mod him down. In fact, you could in theory set up a dynamic system that continuously adjusts the weights of reviewers based upon how well they match your opinions.
A note on resolution: If you're able to get tristate or better "resolution" in your reviewers, more power to you! In fact, I encourage this. However, on a practical note I think it will be difficult to find enough reviewers with a high enough common denominator. Of course, this does not prevent you from assigning special weights to the differing rating systems used by various reviewers. Be creative! Invent your own system. :)
Pipe dream: It's my personal pipedream to have a website where everybody can register their opinions on various topics. Each person could then seek out (or be matched to) other individuals with similar tastes. People with less time to devote to reviewing things would defer their opinions to others. Eventually this would trickle up to a small set of individuals making recommendation
One thing that I have found useful... (Score:1)
Reviews matter? Sometimes. (Score:3, Informative)
I have a few sources that I trust pretty well. But even *those* it's not what score they give it. It's what they say.
The author is guilty of what he's writing about (Score:5, Interesting)
Games should always be considered on their individual merits, on the qualities that they offer and the accomplishments they boast. This can never be distilled into a percentage or ranking out of ten. Hold games up to examination and this evidently becomes the case. For instance, when I reviewed San Andreas I gave it a 90%+ review score. I would not on the other hand award as high a score to something like Castle of Shikigami 2 on the Gamecube though personally I feel it is the better game. It would score lower because it is less technically accomplished, far smaller in scope and offers far less variety. I still prefer it however because what it does it does extremely well and when push comes to shove I would rather play it than San Andreas. That's not to say I think it's more accomplished - I simply prefer it.
I'd ask the writer of the article this: why the hell did you rate GTA: San Andreas better? This IS the problem with these scores. GTA gets a better score simply because the conventional wisdom says it is a more accomplished game, and NOT because the reviewer actually likes it better. He admits it in the article for all to see. Hype = high scores, and even someone who is writing an article about how the scores don't work is swayed by it.
This is how a game like Katamari Damacy gets lost in the Half-Life 2s and Halo 2s of the world. Conventional wisdom says that a strange Japanese game with no real storyline, blocky graphics, and simple gameplay is not as "accomplished" as a sci-fi FPS. The $20 price tag alone almost screams "inferior game." But an expensive price, polished graphics, long development cycle, sweeping advertising campaign, and a big booth at E3 are not what makes a good game.
Re:The author is guilty of what he's writing about (Score:2)
Not saying it isn't a good game, but it sure as heck isn't 'lost.'
Re:The author is guilty of what he's writing about (Score:3, Informative)
Re:The author is guilty of what he's writing about (Score:2)
Just a possibility, I haven't played either game.
Re:The author is guilty of what he's writing about (Score:2)
I actually haven't played GTA:SA or GS2, but I can say that the review should be based on what the reviewer prefers. He said that he prefers GS2, so it should have gotten a higher score from him. Rewiews are too much about what the rewiewer thinks we will feel about a game. I
Get rid of score inflation (Score:5, Interesting)
Similarly, game scores seem to evoke this feeling among fans of particular games. Anything below an 8/10 is perceived as "crap."
In reality, I own games that I would rate as a 6/10 which are still enjoyable. These games may be merely average, but if certain aspects are present, they can still be anjoyable. "Buffy The Vampire Slayer: Chaos Bleeds" would fall into that category. The game received in the 6.5/10 range all over, and it's a score I would agree with. The camera is lousy, and the controls are inferior to the original in almost all respects. Despite this, the story is entertaining, the voice acting is pretty good (with the exception of the knock-off Willow), and the subject matter is entertaining to me. It is a 6.5 game, and I don't believe anything to the contrary, but it's still entertaining.
Dead or Alive 3 is another great example. It's probably a 7/10 game. The graphics are beautiful, yes, but the game wasn't really substantial change from DOA2. Weakening the counters improved the battle system, but the new characters were universally dull (except for Hitomi), and the game was otherwise nothing more than the second. It doesn't really deserve anything spectacular as far as scores are concerned, but it's a favorite with my friends and I when we get together at my place.
EGM was one of the few magazines I discovered that was willing to make this stand. a 5/10 game was AVERAGE. You might enjoy it if it had a particular point that really appealed to you. If you were a huge RPG fan, a 6/10 RPG would be worth buying if you'd already finished the last three 8/10 games. The 6/10 was not crap. Games at 3/10 and below were crap. And a game had to be spectacular to get into the 9 range. Unfortunately, people don't seem to be willing to accept that scale; everything needs to be between a 6 and 10. The problem is that it just dilutes the actually worthwhile games. Gamepro was notorious for this. They gave straight 4.5/5 and 5/5 to Starfox64. The game was good, but it was not worthy of that level of score. When compared to something that truly was, it served to make the worthy game's scores "lesser."
Do scores matter? In EGM's case, I'd certainly agree. Back when I still kept up with that sort of thing for professional reasons (I was an assistant manager at a game store), they were generally pretty trustworthy. In a case like Gamepro's, which unfortunately seems to be more the standard than the exception, it makes the scores completely inconsequential. At that point, I learn to just ignore the score and read betweeen the lines of the reviewer's euphamisms.
Re:Get rid of score inflation (Score:2)
The first sentence is a case of an unnecessary comma; it ended up there because I occasionally decide to reword a sentence without correcting the punctuation that was previously there.
The second example is
Re:Get rid of score inflation (Score:1)
The third example has nothing with it as far as I can see. Care to enlighten me on that one?
I'm not the AC who criticized, but my best guess is that there's no antecedent of "it". Not something that I feel is worth griping about in this case.
In the first sentence, I don't feel that the comma is a problem. But the spelling is, and the antecedent of "they" is uncertain.
Obviously, I don't hold people to perfect grammar in /. posts-- myself included, as you can see.
Re:Get rid of score inflation (Score:1)
You say a game should score 6/10 if it is really fun but has lousy graphics, controls or whatever extraneous detail. I only read the word "fun" and would give it 10/10 if it really is that enjoyable even if the graphics and such sucked. Why? Because you'll enjoy it anyway!
I sure hope you score your students' papers on content rather than the font they use.
Re:Get rid of score inflation (Score:2)
A game with little replay value is still capable of being fun while you play it. Is it worthy of a perfect score?
A game with frustrating controls is still capable of being fun. Is it worthy of a perfect score?
My point there was that I'm still capable of enjoying a flawed game, just like I'm still capable of enjoying a flawed movie, book, or even a website that routinely posts duplicate new
It depends on the source (Score:2)
The scores aallow me to gauge whether something is worth reading up on more or not. If a game is 80% or higher, then I like to see if it's a good fit for me. The scores provid
Re:It depends on the source (Score:1)
(This is opposed to the silly idea that a rating be
my magazine (Score:2)
Over the last couple of weeks I've been scratching my head at the score system I've been using and asking others to use. Invariably, nothing is ultimately compatible. And everyone is biased.
So the right answer here is that I'm actually going to remove scores from all the game and comic reviews.
That will actually force readers to REAd the mag. What a novel concept.
Re:my magazine (Score:1)
Re:my magazine (Score:2)
If they want a magazine to just look at the pictures, then they should be buying a porno. I've made sure that mine is less outright review and more article.
we'll see what happens. If people decide that they want scores, they'll tell me. Or not buy it. Hmm..
Don't bother to RTFA (Score:1)
;-)
Oh the irony (Score:1)
If I'd looked ahead I could have skipped it instead of wishing I had 3 minutes back.
He says it himself (Score:2)
FPS is a particularly good example - games like DooM3 and HL2, both excellent FPS games with their own strengths, reviewed very highly compared with games like Sniper and The Thing (full of bugs). As a result, if I go to gametab or another aggregation
Re:He says it himself (Score:1)
However, I do believe that numerical ratings have their place as a measure of how strongly the reviewer recommends the product. An earlier post [slashdot.org]) was on the right track in terms of using ratings as a measure of recommending the game, t
I think (Score:1)
Re:It depends on the magazine (Score:2)
Really, take one year of a magazine having such a system and look at all their reviews and see were most of them are. I can tell you it is not in the under 70.
In addition, the absolute main focus, i
Try it yourself (Score:1)
I get much better reviews from regular users than from 'the professionals' (i.e. 'This game sucks, the controls are terrible', 'I played this game in 2 hours, it's too short!', 'This is the be
Re:Try it yourself (Score:1)
You're lookin at it all wrong, schee (Score:1)
Computer Games Magazine (Score:2, Informative)
Woeful name for a publication, however, I perceive their reviews as being extremely honest and balanced. They won't hesitate to pan a game, and I definitely don't get the feeling that they are giving a bia$ed apprai$al. Their coverage is detailed and offers a refreshing maturity compared with most of the magazines that
Xenon 2 (Score:2, Troll)
When it came out in the UK one gaming magazine (I think it was CVG) gave it 100%.
I wonder if that reviewer is still playing it today, as it was obviously the best game ever. It must have been impossible to get tired of the game if he scored it 100%, right?
RM
Re:Xenon 2 (Score:2)
Just look at all the reviews from last Xmas... (Score:1, Flamebait)
Anyone who reads a game review to make a buying decision needs their head examining IMHO. The reviewers give the scores they're paid to by the big publishers.
These days I decide on a game by either playing a demo or looking at screenshots and movies and seeing if it looks interesting.
The score should be irrelevant (Score:1)
If one reads a review and the reviewer is making the game sound great, and then there's a score of a 4/10 stuck on the end (or whatever), that's because the reviewer didn't do a good job of explaining themselves.
Too often with reviews, though, people don't even bother reading the review, and just go by the score. In my mind, reviews in general would be much better if scores were completely omitted. That way,
They totally matter! (Score:1)
Slashdot gaming poll? (Score:1)
Re:Slashdot gaming poll? (Score:1)
Game Reviews are No Worse than Film/Music Reviews (Score:1)
If we pull back and look at the big picture, we can see that a lot of the same issues are raised, or example, against film reviewers (biased, paid-for reviews, etc.). But film criticism is still held in higher regard than game reviews - largely because film critics are most often journalists first, while game writers are gamers first, journalists second (often a distan
My 1-2-3 method of finding good games (Score:1)
2) Click on my gaming system of choice.
3) See what the current Top 10 FAQ Pages consists of
More often than not, if a majority of people are looking for a guide for a given game, they're looking for it because they're enjoying it.