Website Reviews and You: Part 1

Unless you've been hired as a Q/A consultant by every major hardware manufacturer out there, then chances are you're not in a position to make a purchasing decision based on hands-on experience.  Sure we all have a select few companies that we're comfortable working with based on past experiences, but the real beneficiary of brand loyalty is the brand itself and not you, the consumer.  Yet, that's how the game was played a little over a decade ago, and if you were building your own PC (far from the affordable endeavor it is today), you had to rely on past ownership, word of mouth, and marketing.

And then the internet was born, and among other things, it brought the DIY (Do-It-Yourself) crowd a vast resource for making informed purchases rather than relying on the bullets on the box.  Wondering how that new Asus motherboard stacks up against the competition?  Google it!  Does ATI still offer the best bang for your buck, along with the features you need?  Hit up your favorite review sites!

But what really goes on behind the scenes at online review sites, and can you really trust what you read?  What's it like being a hardware/software reviewer?  Do the chicks dig it?  If you find one that does, she's a keeper, but that's beyond what I want to talk about in this blog.  Instead, I want to focus on some common FAQs, and perhaps some not-so-common, yet still important, issues you should be aware of when you're reading that online review.  What better way to do that than to jump right in...

Why do you have banner ads?

There's also the cost of keeping test equipment up to date.  No one wants to read about the latest high end videocard if the testbed in question is two generations old and an obvious bottleneck.  We sometimes use review samples to keep our testbeds current, and other times it's an out-of-pocket expense for the site.

And then there's the benchmarking software.  Some companies are willing to grant review sites software licenses in exchange for the exposure they get each time a product is reviewed, but not all software is free.  Nor do we (speaking for HardwareLogic) build our benchmarking philosophy around which software we can get at no-cost, and instead we use what we feel is the best product for the job.  For example, we just recently added Supreme Commander to our gauntlet of gaming benchmarks, and as time goes on, we'll be adding even more current titles to the lineup, and replacing some older ones (like Doom 3).  At the going rate for most major videogame titles, that equates to roughly $50/pop for each reviewer that requires a copy.

That, folks, is why we have banner ads.  We're not fanboys of any particularly company, nor does an ad insinuate an endorsement on our part.  Just like a magazine, the ads help pay the bills, allowing us to exist.  If you're happy with our work on HardwareLogic, consider disabling AdBlock or any other ad-blocking software you may have installed when you visit our site.

So the ads pay the bills, but do advertisers play a role in your reviews?
Absolutely, positively, emphatically NOT!  Advertisers are paying for exposure, and not for favorable or lenient reviews.  We'd rather lose an advertiser than compromise our credibility, and having built relationships with several other review sites around the web, we don't feel we're the exception when it comes to this concept.  Unfortunately, that's not to say that there aren't exceptions, and some companies do get downright pissy in reaction to an unfavorable review.  We're not going to point out names, at either websites or companies, but we do strongly suggest reading online reviews from multiple sources, and read them carefully.  In fact, almost every review we post on HardwareLogic will list links after our conclusion to several other write-ups on the same product.

Do you get to keep review samples, and if so, what do you do with them?
In most cases, yes, we do keep review samples.  There are exceptions, such as OEM systems, hard drives, and other odds and ends that need to be returned to the manufacturer after reviewing them, but by and large, we keep most everything we review. 

What we do with them afterwards is either upgrade our testbed or personal machines, give them away to staff, or give them away to you, the reader.  Long time members of HardwareLogic are well aware of the amount of hardware we've given away since our inception, whether it's for Forum Member of the Month, Forum Member of the Year, contests, or simply to help a forum member or family in need.   In  fact, I'm not aware of another tech website that's given away as much swag as we have, or as focused on good will, and that's not something we plan on changing anytime soon.  Rest assured we didn't fund our sports car with review samples.



Can you write an unbiased review knowing that you're getting a free product?
This is almost always a challenge for new reviewers, and there's definitely a learning curve involved.  At HardwareLogic, all reviews go through both myself and site owner Rich Caporali before posting on our main page.  Most of our staff is seasoned enough to avoid this pitfall, and the ones that aren't quickly learn the ropes from what we expect.  But that is something you need to be on the lookout for when reading an online review.

If a review contains a laundry list of complaints and follows each one up with an excuse ("The videocard overheated constantly, but we were playing an intensive game," "During overclocking, the system would restart constantly, but Core 2 Duos don't need OCing to run fast," etc), then weigh carefully what's being said against what's being reported.  In the first hypothetical example, what's being reported is that the videocard has a tendency to overheat, but what's being said is a justification for the product's flaw.  While that could be sign of catering to an advertiser (discussed above), it could just as easily be the sign of an inexperienced reviewer, and/or one that's letting a free review sample create a biased assessment.  In our opinion, it's okay to forgive certain flaws based on the product's target audience (we have stricter criteria for a $300 premium computer case than we do for a $25 Wal-Mart special), but it's ultimately up to the manufacturer to sell the product; our job is simply to review it.

Tell me more!
Whoa, not so fast eager reader!  We're just getting started, and there will be a part 2 that dives deeper into the review process, just not today.