About Us | Game Reviews | Feature Articles | Podcast | Best Work | Forums | Shop | Review Game

Reviewing the reviewers

Mike Doolittle's picture

Halo 3 Screenshot 

When Bioshock was released to universal acclaim from the popular gaming press, I fully expected to be amazed by the game. My reaction to it was less enthusiastic than what I expected, and reading around the 'Net as well as the responses to my own review, there seemed to be a similar reaction among many others. I found it possibly more than coincidental that sites like IGN were loaded with hands-on previews and detailed articles about the game, then it was released to an exceptionally praise-filled review. Now the long-awaited Halo 3 reviews are in, and—just like Bioshock—there were tons of hands-on impressions, detailed previews, and fever-pitch hype. So it didn't surprise me that it received ubiquitously raving reviews, with 9s and 10s littering the field.

I haven't played Halo 3, and I probably won't play it until it comes out on the PC three years from now. But I read the reviews and I have yet to read one that I thought really justified the high (often perfect) scores the game is receiving. A 10/10 is, to me, a game that is truly a landmark in game design. It breaks boundaries, drives the medium forward, and executes brilliantly across the board. A few years back I gave The Chronicles of Riddick: Escape from Butcher Bay a 10. It was a beautiful and brilliant game that melded numerous genres into a fluid, exciting, and wholly unique experience. From what I can see, no one is claiming that Halo 3 does anything like that. The consensus seems to be that the gameplay is mainly just a minor refinement of Halo 2; that there are some level design issues later in the game; and that the experience lacks the "newness" of Halo. 1UP.com even went so far as to say, "...in Halo 3, the big 'oh wow!' gameplay moments just aren't there"—but they gave the game a 10/10 anyway. Now, having not played Halo 3, I can't say personally whether it's as great as it's being made out to be—I'm only saying that the press has, in my view, done a poor job of substantiating their ratings.

All this leads me to wonder some things about the gaming press. Most commercial sites are hands-on participants in the pre-release hype. Developers give them exclusive stories; they write detailed, often spoiler-filled previews that draw lots of readers and fuels the hype; developers often invite them in-house for multiple demonstrations and hands-on sessions with early builds of the game. I find it hard to believe that these kinds of sneak-peeks do not entice and excite the people in the gaming press and cause them to fuel the hype among eager gamers. And I find it even more difficult to believe that these kinds of teasers, along with the self-perpetuating hype, do not eventually sway the biases of the writers when the game hits.

Well, I did not get hands-on time with Bioshock at E3, or visit Bungie's studios to get a sneak peek at an early build of Halo 3. The gamers' reaction to the former was most telling—while most seemed to enjoy it, it seemed that few viewed it as the masterstroke of game design that the press portrayed it to be. I suspect that the reaction to Halo 3 will be similar; I somehow doubt that gamers—at least experienced ones—will be quite as awed as the press.

What concerns me most about the hype slant, though, is that better, more creative games are often overlooked. My favorite game this year has been S.T.A.L.K.E.R.: Shadow of Chernobyl. While certainly not a "perfect" game, it was a brilliant and gripping game that broke new ground in a number of important ways (I'm very much looking forward to the upcoming prequel, which looks to expand on some of the key innovations in the game). I remember Riddick, which was released with little pre-release hype, being largely underrated by the press but being highly praised by gamers everywhere.  I'm also looking forward to Crysis, which I'm sure will get terrific reviews and be subject to a hype slant itself. But it's clear to me from the previews that the developers are really pushing technology and interactivity in some really groundbreaking ways, so when a game is released that is "merely" a refinement of well-played ideas, it seems to be a bit of a disservice to developers who are really pushing boundaries.

Category Tags
Articles: Editorials  

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

My goodness!

Now, Mr. Doolittle, I have just read your review of Bioshock and commented on it, and then stumbled on this entry of yours.

With people like you there seems to be hope for the gaming press.

Just don't get hired by the usual suspects.

yep

I wouldn't be surprised if just about every Halo 3 review expresses more than a little disappointment with the game, while giving it an uber-score regardless.

It's curious and disappointing that the printed gaming press' convention of giving BIG games exactly what they think their readers want them to get has been carried over so blindly into most web-based magazines. I guess they've got just as much to lose from giving unpopular opinions at the end of the day.

Suggestion for 1-100 score scales:
Gauge the hype and coverage of a BIG game from 1-9, times it by 10, then judge the site's real opinion of that game based on whatever it gives it between the multiplied hype number and 100. Example: IGN gave Halo 3 95, which means 9(max hype) x 10 + 5 (out of 10; how they really feel) = 95. Hmmm....

hmm ok so you are passing

hmm ok so you are passing judgement on the game without playing it? yup you are much better then the other reviewers

No my friend

No, I didn't make any critical remarks about Halo 3. As the title of the blog indicates, I'm addressing the critics. When a review says that there are AI problems, level design problems, feelings of sameness and routine, minor gameplay refinements – and then gives the game an astronomically high score usually reserved for near-flawless, groundbreaking games – I have to question whether the reviewer is being swayed by the hype. I don't have any merit to criticize Halo 3, but I have the merit to judge whether I feel that critics are justifying their ratings with strong arguments.

I can't judge on Halo 3,

I can't judge on Halo 3, having not played it or indeed Halo 2. But you do seem to be on the money re: the disparity between scores and the tone of the review. To me this indicates that maybe (like this site) it's time to move away from arbitrary ratings.

I do have to disagree about Bioshock. I thought it was every bit as amazing and compelling as it was talked up to be - by the reviews. The previews discussed something that could have been even more exciting, but then that's why I try not to read previews.

And I seem to recall a lot of rave reviews of Chronicles of Riddick, even if it doesn't seem to have stuck in people's minds quite as much as games I personally liked significantly less. 90 on metacritic would seem to bear that out. (Though it's a few points less than Bioshock and Halo 3, it's also been out long enough for a wider range of people to weigh in. I wouldn't be surprised to see one or both of the aforementioned drop further points as time goes by. Hell, I think Bioshock was hovering around 99-100 in the first few reviews.)

Riddick wasn't that great,

Riddick wasn't that great, though. It was pretty decent, and there were some parts of the game that were excellent, but overall, I personally don't think it was a 10/10.

For one thing, it was far too short. A typical gamer could finish it in under 8 hours. This is including trying to find all the secrets and exploring every nook and cranny.

The graphics were decent for the most part, but the engine was a bit too much for the original Xbox to handle, and it was framerate locked. This caused the level of detail to go up and down quite drastically as it struggled to maintain the framerate.

Finally, there wasn't much replay value. The game was pretty linear and once I finished it, there wasn't any incentive to go back and do it all over again.

Riddick

Riddick melded a number of gameplay styles and executed each superbly, had a great story, huge open-ended levels that could be explored for hours and allowed multiple paths toward your goal (if you didn't experience this, you haven't spent enough time with the game), had arguably the best first-person melee system yet developed, and had the best graphics and sound on the XBox (it was the first game to use normal mapping). Yes, it was short, but it was nonstop. And there is tons to do when you replay the game, as it's likely that if you completed the game quickly, you only explored a fraction of the later levels.

The single player was praised by the press and it scored fairly highly, but it didn't get the Halo 3 kind of scores because it had no multiplayer. That didn't seem to hurt Bioshock or Half-Life 2, but Riddick didn't have the hype on its side.

I completely agree with you

I completely agree with you about video game journalism. I don't think it has ever been objective. And over-hyped games like the Halos really help to illuminate the disparity between the reviews and reality.

And you're right that the reviews for Bioshock were off pace. But so is yours. Bioshock's innovation lies with its narrative. It played within the confines of the standard FPS gameplay in order to tell a mind blowing story. The story wouldn't have worked as well if they had tried to make deviate too much from a standard FPS.

Riddick?

Anyone giving riddick 10/10 needs to have his head checked. If Riddick is a 10, Halo 2 is a 12.

Riddick might have done some new things, but it certainly had quite a lot of issues otherwise!

I'm not sure how you can

I'm not sure how you can "review" reviewers of a game you haven't even played. It's not a total engine overhaul, but it puts all the pieces in JUST the right places, and does it with far more presentation/stats/details/preferences/gametypes than any game out there. I'm not even a Halo fanboy, I just know a good game when I play it.

Now you've done it

I agree with you wholeheartedly about the big gaming press. Reviewers seem just as, if not more, susceptible to marketing hype. In addition, they tend to preach that hype from their well-funded mountains to the detriment of those searching for an honest opinion. I think it goes beyond disparity between copy and numerics, and is often woven heavily into the reviews themselves, but I'll leave that argument for someone else to make.

However, I've had an unopened copy of Riddick sitting on my shelf since it came packaged with my video card a year or two ago. I'd heard it was decent, I just never got around to playing it. With a 10/10 from the man who gave the first real review of Bioshock I'd ever seen, I'm going to have to give it a shot. Your rep is on the line, buddy. :P

Addendum

I'd like to add just a couple of things at this point. First is that the heart of my criticism is based on my own bias – that a 9.5 or 10/10 game should be a brilliant, groundbreaking game. Not a "merely great" game with lots of polish, but one that really raises the bar for all games to follow it. Perhaps others feel that a 10/10 game can be one that is just really great and polished, even if it doesn't do anything particularly new or groundbreaking in gameplay or design.

The other point is that it's important to draw a distinction between criticizing a game and criticizing an argument. A review is an argument. If someone reads a review I wrote, you're certainly welcome to disagree with my opinion of the game; not everyone is going to like all games equally. But I would hope that you would at least feel that I supported my opinion well. My job as a critic isn't to appease readers, but to present a strong argument and support my opinions with meaningful reasoning. Because it's one thing to say, "I disagree; I think the game is better/worse than you say it is" versus saying, "I don't feel you made a convincing argument as to why the game is good/bad." The latter is what is really the point of contention I'm addressing in this blog. I'm a huge fan of the first two games (which I scored a 10 and a 9, respectively), and for all I know Halo 3 could quickly become my all-time favorite game. But the issue for me is how my peers in the review industry approach their craft, not whether Halo 3 is any good.

A couple of the anons seem

A couple of the anons seem retarded. He didn't criticize the game itself, so playing the game is irrelevant. He's obviously criticizing the disparity between the written reviews and numerical scores given by game journalists. A 10/10 has gone from "Holy crap you have to play this game, it's amazing!" to "Yeah, this game's pretty good."

I stopped caring about game scoring a while ago because it's so misleading. I just read the reviews themselves and not just from journalists but average players as well. The way I see it, scores are for the dumb masses that feed off hype and written reviews are for people who actually care about the quality of what they're buying.

Why do game reviews have

Why do game reviews have scores? I can think of a few possible reasons.
1. To get a blurb on the box.
2. To show up on sites like Rotten Tomatoes and Metacritic.
3. To satisfy the obsessive types who create top-10 lists of their favourite things. (See the movie High Fidelity for examples)
4. Top-10 lists are a lazy and effective way to make a feature article when there is no news in the summer.

The worst offenders are those who give individual score for "graphics", "audio", etc. and tally them up - as if they are reviewing a car or grading a math test.

Anyone seen the movie

Anyone seen the movie Metropolitan, about these unbelievably pretentious college kids? Pretty funny movie, and the main character gives this great speech about how he doesn't read novels anymore, just literary criticism, which contains both the story and the writers opinion.

Doolittle, don't be that guy. It's just sad. If you want to say anything about the reviews or scores of Halo 3, you're going to have to at least play a couple of hours worth.

You should take down the

You should take down the picture of Halo and put up a picture of a review of Halo and then maybe some of these people will understand what you're doing here.

The main thrust of this

The main thrust of this article is that game review scores may be skewed by hype and press junkets. Well that's hardly news is it?

You also haven't considered that some reviewers overstate or understate things as part of their style. A 10/10 from Gamecritics isn't the same as one from Adrenaline Vault or 1up.com, for example.

What's interesting is that the hype surrounding Halo 3 is comparable to the hype surrounding the Halo 1 based on similar stated benefits - more refined graphics, physics, AI, etc. You just can't pull the same trick three times!

I haven't read this yet, but

I haven't read this yet, but I couldn't agree with you less.

I haven't read this

I haven't read this http://www.popmatters.com/pm/features/article/48962/rethinking-halo/ either, but the hype suggests it might be ground breaking - especially the way it refers to Mike Do Little.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Code of Conduct

Comments are subject to approval/deletion based on the following criteria:
1) Treat all users with respect.
2) Post with an open-mind.
3) Do not insult and/or harass users.
4) Do not incite flame wars.
5) Do not troll and/or feed the trolls.
6) No excessive whining and/or complaining.

Please report any offensive posts here.

For more video game discussion with the our online community, become a member of our forum.

Our Game Review Philosophy and Ratings Explanations.

About Us | Privacy Policy | Review Game | Contact Us | Twitter | Facebook |  RSS
Copyright 1999–2010 GameCritics.com. All rights reserved.