On the Integrity of Hardware Review Sites 263
leathered writes "Charlie Demerjian of The Inquirer has posted an interesting article on the integrity of hardware review sites. Apparently the benefits of running such a site go far beyond advertising revenue with a fair amount of 'sweeteners' from the hardware manufacturers to say the least. All is not lost as Charlie informs us that there are a small number are flying the flag for trustworthy reviews, but the question of which sites we can trust remains." I like Daniel Rutter's (of Dan's Data) policy best.
Trust? On the net? (Score:5, Insightful)
But, then again, how do you know I'm not just making this up?
Integrity? (Score:5, Insightful)
Re:Integrity? (Score:5, Insightful)
The first PC I built from scratch was in college (my previous computers were either my parents or a small-shop one I bought). I hit the hardware sites and many claimed that a particular ASUS board was great and rock solid. I was naive and took the 3 or 4 sites word at it and bought it. I started having MAJOR problems and later found out it was the Via chipset on the board. Forums were FLOODED with the exact same complaints. I eventually had to replace the board.
Since then I don't buy hardware until I've searched forums for personal experiences with said product. I'll still look at some review sites but I now take what they say with a grain of salt.
Im not sure if he understands multithreading (Score:5, Insightful)
Correct me if im wrong but isn't multithreading/multitasking pretty damn important considering all the background tasks/services that are needed just to keep an OS running?
Makes Sense (Score:2, Insightful)
Re:Integrity? (Score:3, Insightful)
Re:Trust? On the net? (Score:1, Insightful)
Re:Trust? On the net? (Score:3, Insightful)
This is not journalism (Score:5, Insightful)
Re:Im not sure if he understands multithreading (Score:1, Insightful)
Crux of the problem (Score:5, Insightful)
Without this hardware you have to buy your hardware yourself. Not only is this expensive, but by the time your review is out the ad value of your review of bleeding edge hardware is kaput. Unfortunately, these are the ones that do the most honest and best reviews. Pre-release hardware is often picked out from a large selection to make sure that the review site gets a good "sample". These reviews are also the least profitable for the review sites to do. It's a nasty catch 22.
"I take goods you send me for free & review"? (Score:5, Insightful)
We accept no advertising, and buy any products we test on the open market. We are not beholden to any commercial interest.
I'm sorry, some guy who writes reviews, even ostensibly fair ones, in exchange for free product can't stand up to this.
what if everyone's in on it? (Score:5, Insightful)
What good is reading multiple reviews if they're all crap? What good are end-user experiences posted on the net, if companies are posting fake reviews, which they are?
News flash- even if they're not getting "payola" (let's call it what it is- bribe money/gear), they're controlled quite effectively by hardware companies because everyone wants to be the first site with a review of Hot Product X to drive hits to their site to earn advertising revenue. Write something bad about a product, and that company will drop you to the bottom of the list.
Let's not forget that most of these guys litter their sites with advertisements for the very product they are reviewing, too. Bob's Extreme Hardware isn't going to be very happy if young Johnny says the PC case Bob just stocked is crap- and he's going to tell young Johhny that.
Why is any of this a surprise to any reasonably intelligent individual?
Re:Trust? On the net? (Score:5, Insightful)
I hope that includes Charlie Demerjian. This jumped out at me:
"here is the truth, if you are going to multitask and do and do anything that tasks both of the CPUs, one of those is going to be a game."
Bullshit. This drives me crazy on hardware sites, this supposition that the only reason anyone could ever want high performance in their PC is to play games.
At home, I use Photoshop, Illustrator and Flash, I run Windows Media Center for SDTV and DVD viewing, I do video encoding using various tools (Windows Media Encoder, Dr. Divx, and others). I often do all these things at the same time on the same PC, with a hacked version of Media Center that lets me log in remotely at the same time another account has the TV going.
I studiously avoid playing games on this system, because I'm asking it to do quite enough already - I've got another system that I play games on. But I would love a dual-core CPU for this thing, as it would help me out a lot.
Graphics professionals, photographers, multimedia content producers and other high-end users are, surprise surprise, a real market, and they spend even more money than gamers do. I don't see why that's so hard to grasp. I read the specific preview of Intel's dual-core CPU's that Charlie's talking about in his comment up there and I actually found it a refreshing change to find some real-world benchmarks that were not strictly based on playing Doom 3.
That said, I'm sure there is payola going on in the industry. But I worry more about the small sites that seem to give positive reviews to every single component they get sent for free than I do about sites that realize non-gamers are a legitimate group of users that require their own set of benchmarks.
Re:This is not journalism (Score:2, Insightful)
Re:Two Words (Score:2, Insightful)
So if you intended to say the article was talking about toms then I'd say you're wrong.
If you were saying toms is a shining light in a sea of intels bitches, I'd say not far off.
Re:This is not journalism (Score:5, Insightful)
This will encourage people to check with multiple sources, talk to people, and make product decisions for themselves, which is possibly one of the most important skills one can learn for surfing the internet.
might be true, but... (Score:2, Insightful)
Article is Flamebait (Score:5, Insightful)
His only criticisms of the review are that it was an exclusive (which the article makes clear) and that it doesn't cover gaming (although it is only the first part of the review). He himself admits that gaming is not the point of these chips, so why does he feel that Anandtech should have to focus on gaming is the first part of their article? Indeed, in their second part they do cover gaming and conclude that you should buy an Athlon 64 if you mainly play single-threaded games, a fact that would be obvious to anyone who regularly reads any hardware site.
I can't claim that the hardware review sites are all without bias, but compared to mainstream news, hardware reviews are some of the hardest to bias given the ease of doing standardised, repeatable benchmarks.
Reviews are mostly BS (Score:3, Insightful)
If I'm in doubt about something, I'll read reviews from actual people, e.g., at newegg. When someone gets screwed over a product, they aren't going to gloss over the problems. They are going to tell us as bluntly as possible.
Re:Two Words (Score:2, Insightful)
Re:Trust? On the net? (Score:4, Insightful)
Agreed.
I don't play computer games, I don't even *own* any. None of my systems have ever had a game installed on them (yes, this includes solitare). All of my systems are used for work, and non-work related research. Yet, over half of my systems are dual-cpu. I multi-task, a lot. I often have graphics filters chewing away, or CDs burning, while I'm doing something else. On my servers, I want to know that if I need to compile something, archive something or do some other processor-intensive task, that there's enough processor power left to continue with the server's normal tasks.
I read reviews, but they account for less than 20% of the weight I give to my purchasing decisions. The only reviews I will pay close attention to are the *bad* reviews. They're so rare that one has to believe the product must have been truly hideous.
Re:Trust? On the net? (Score:4, Insightful)
Graphics professionals are indeed a market, but professionals of any stripe generally use the hardware their company procures for them for that profession (I know I do). The gaming machine is the one you build yourself from parts. The gaming bias of the review sites makes a lot of sense in that respect.
What I do... (Score:3, Insightful)
Through all the data that I read, my brain forms an opinion and I weed out the bullshit and the hyperbole to find the heart of the matter. It usually works most of the time too.
Your bullshit detector has to be in good shape and you have to know how to weed out the crap before you get down to the nitty-gritty.
But if you're naive about the net and you go online maybe once a month...then you're a raw piece of meat in a pool full of sharks.
It's the meta-information that's important. (Score:4, Insightful)
Say it ain't so!!! (Score:3, Insightful)
My experience with both links (Score:4, Insightful)
As for the main article, I am surprised how badly it is written. Poor language use and overall structure.
You get what you pay for. (Score:2, Insightful)
Re:Two Words (Score:3, Insightful)
Priceless =)
It's more than just that... (Score:3, Insightful)
If you actually BUY the product you are reviewing, you're subject to the same likelihood of manufacturing defects, poor workmanship or shipping/transporting accidents as anyone else who buys it -- so you can work that into your review to give an overall impression of what a REAL consumer will face if they buy the product.
If you're testing a product that was specially given to you by the manufacturer, they want as favorable a review as possible, so your product may have been specially checked for defects, or have extra durability built in, and it will likely arrive insured by special courier or some other express guaranteed delivery -- not representative of the average customer experience at all.
IMHO, Consumers Union (publisher of Consumer Reports) is THE authority when it comes to honest, reliable and unbiased reviews.
On the integrity of the Inquirer (Score:3, Insightful)
This is the same site that called a Microsoft representitive the "spokesvole".
The article complains that Anandtech (it's obvious for anyone who read the review in question) doesn't use any gaming benchmarks.
Well, take a closer look.. the article is called Intel Dual Core Performance Preview Part I: First Encounter
very conveniently, (and completely expected, as well..) Anand posted the second part today, which included the gaming benchmarks we all expect.
Way to troll Inquirer writer, way to troll..
Re:Integrity? (Score:3, Insightful)
I won't buy a major part until I've specifically found some problems with it. Nothing is perfect, and until I feel I've got a reasonable sample of the sorts of problems a part has, I'm not comfortable buying it.
Besides, the stuff I do is generally improved by throwing gobs of RAM at the problem. That's generally easy to do no matter which parts you're buying.
Re:Identification of sites he's accusing? (Score:1, Insightful)
Think about this rationally for a minute: do you honestly believe it is possible for a 3.2GHz dual-core processor to outperform a 3.8GHz single-core processor in a single threaded benchmark? Me either; it's impossible! But that's exactly what you're doing when you make a performance comparison of the two processors using a benchmarking suite consisting entirely of games. Review sites that make this kind of comparison and draw the conclusion that dual-core is a waste of time have _completely_ missed the point; this is NOT what dual-core processors bring to the table.
The introduction of dual-core to the mainstream is not supposed to compete head-to-head with the best and fastest in GAMES. I mean, who gives a fuck about games? Really! Are gamers the only market out there for PCs? I highly doubt it. The sad fact is, according to most review sites on the 'net, PC performance equates to game performance. The introduction of some very exciting technology (essentially the SMP system I've always dreamed of owning on a single piece of silicon) is met with luke-warm response by the mainstream review sites because it doesn't smoke an FX55 in Half-Life 2...
AnandTech published an article that focused on the multitasking performance of dual-core systems, because... wait for it, people... multitasking performance is what multi-processor systems are designed for. The preview was not bought by Intel or any such bullshit; it wasn't designed to paint the new dual-core processor in the best possible light. The author, unlike the rest of the mainstream review sites it seems, realized, in advance, that the benefit of running a dual-core system lies in the silky-smooth multitasking performance it offers. Anand requested input [anandtech.com] from the community in an attempt to gauge what the average user's multitasking workload looks like, and put together a suite of benchmarks based on the feedback received from real users. It amazes me that the only article published to date to show me -- in real world terms -- the kind of performance benefit I can expect to achieve from a dual-core system and, more importantly, where that benefit is to be realized, is met with such harsh criticism. It disgusts me that the Inquirer would publish an article that clearly attacks the journalistic integrity of another site, without first doing the research to back up their claims; just two minutes researching the article and the community input used to produce it would have negated any evidence of a buy-out by Intel.
So my question is: who's journalistic integrity is jeopardised now?
(why don't I have an account? Well, I don't post here often enough to bother with one... I hate this fucking site. Eat me.)