I've always teased friends at CNet about their rating system, which appears to always rate products between 7 and 8, meaning everything is more or less "very good" in score. Ecoustics has done the job of analyzing 1,325 reviews from 2007 (all of them?) and figured out the exact math: 96% of all ratings from last year fell between 5.0 and 8.9; about 70% fall between 6 and 7.9. Maybe CNet should make anything a "CNet 6 or below" a "1", and anything that scores a "CNet 10" a "5". After all, anything below a 6 means DO NOT BUY to me.
Other interesting points:
8.0 to 8.9 Excellent A product that receives a rating in this range is superior in so many ways that its relatively few drawbacks are not very important. 18.6%
7.0 to 7.9 Very good While the strengths of a product scoring in this range certainly outweigh its weaknesses, it has some minor faults that certain users should be aware of. 41.8%
6.0 to 6.9 Good This range represents a product that is above average. Its strengths slightly outweigh its weaknesses, making it good for most uses but not a standout. 27.8%
So most products rate as very good. One might argue that "Very good" means average, and if 40% of all gadgets are rated so, they should be called that. ("Average" is a 5.0-5.9 on the CNet scale.)
It is, of course, complicated. Average implies, at the bottom line, that you probably won't be thrilled to own a device with such a CNet score. While "Very Good" implies you will be. Regular people will be happy with a lot of this gear, while the best gets an "Editor's Choice."
It is interesting that the video game reviews, which are much more subjective, made up the head and tails of the reviews.
The highest and lowest rated products both happened to be video games:
* 9.5 - The Elder Scrolls IV: Oblivion (PlayStation 3)
* 1.7 - Pimp My Ride (PSP)
CNet's reviews strive to be objective and reasonable, and in this, they've succeeded across the board. But I do wonder if a tighter review scale might serve the public better, along with more opinionated takes on what the best piece of gear in every category is. I mean, how often do your friends ask you what the second best set is on the market?
On top of telling them what I think is the best deal or best overall, I've long made a habit of recommending brands to friends, instead of particular models, so it's also interesting to see that ecoustics did a rundown of which brands did best, with Casio scoring lowest on average and RIM being highest (yes, over Apple.)
So, check out the article. I still use CNet for research and buying advice, and I'm sure many of you do, too, so it's good to understand that rating system. [ecoustics]