How Well Do You Trust Online Reviews?


Image courtesy of XKCD

I recently read a very interesting post by Andrew Leonard about online reviews, and attempts by software companies to (as we say in the high tech world) productize them.  The question is valid:  In an online world with an overwhelming wealth of information about almost everything, how do you weigh reviews and make a buying decision?

Certainly the online world is swamped in reviews.  But in my view, there’s a huge problem with reviews on the Web.  There are almost always just two types:  The rave, and the pan.  That is, either you see posts that say the item is the greatest thing since sliced bread, something you absolutely can’t live without, something you should buy NOW NOW NOW!  Or it’s the worst thing ever, you should avoid it at all costs, and the reviewer wishes he or she had the time to go to company headquarters and burn it to the ground.

In the first case, some portion of the 5-star reviews are by company marketing or sales people, puffing up their product and artificially ramping up their average review score.  The rest are written by random people who love the item, but rarely give you any good info on whether you might love it.  A 5-star review from Random Guy is basically useless.

In the second case, almost always the person is mad simply because of one problem.  The controls don’t work like they want them to, or they had a bad support experience, or some other minor problem.  Again, they frequently don’t have any solid information, and don’t give you an idea why the item is only a 1-star product.

So what to do?  Over time–and I’ve been online reading reviews since the CLI/USENET era, so I’ve had some practice–I’ve found that there seem to be a couple options that work.  The first one is pretty old-school:  Find a web site and/or reviewers you trust (such as your faithful Gear Diary crew!), and read their reviews of whatever product it is you’re considering.  Consider movies as a model:  How often do you go to Rotten Tomatoes to see what the score is?  And why is that?  Because the reviews that they aggregate are from professional reviewers!  Why do you come here for reviews?  Because you trust our reviews to be thorough and informative.  This is what I do most of the time; I search “gear diary review [item]” to see if we’ve reviewed it.  If not, I go on to option two:

The other method is a little counter-intuitive:  Just read the most negative reviews.  Yes, most of them are poorly written, with bad spelling and sometimes barely-concealed explicatives.  Often, they are focused on a single problem–In iTunes, it’s often that a reviewer had an early version of the app that crashed, or they have an old iPhone or iPod touch and the app doesn’t work on old hardware, or something of that nature.  But at least in the negative reviews you can find out about bugs and problems with the item; the glowing, five-star reviews almost always fail to give you any data at all and are thus useless.

Anyway, that’s what I think.  What about you?  Why don’t you tell us your methods for getting online review information below!

Categories: Editorials


1 reply

  1. In the case of Amazon anyways I may read the first listed of the 5 star reviews, but generally skip them and the one star reviews unless there are only a handful. I spend most of my reading time on the 2-4 star reviews which usually have a mix of positive and negative statements and frequently a better Sturgeon’s law coefficient.

    Newegg Is pretty much the same deal though usually I’m scanning those for linux compatibility statements regardless of how many stars the review gave.