Criticism about Criticism

I’m mad about two separate but related things that have to do with media criticism.

A 10 Point Scale is Bullshit

The 10 point scale is terribly common and it’s also terrible. Here’s all criticism of the 10 point scale boiled down to one simple question:

What’s the difference between a 6 and a 7?

If you’re doing qualitative reviews, and nearly every media critic is doing qualitative reviews, then there’s essentially no measurable difference between two adjacent points on a scale. You get the perfect 10, which is never used, and the garbage bin 1, which is also never used, and those are your only absolutes.

The middle of the road changes depending on whether you perceive a 5 to be average or a 7 to be average. A 5 average makes the most sense strictly from a numbers point of view, but the 7 average aligns with a C grade from school, so it’s extremely hard to escape the perception that anything less than a 7 is below average. When you, or your audience, perceives a 7 to be average, then you will use far less of the lower end of the scale, because it doesn’t matter. If it’s not the worst thing ever, but it’s less than average, and your perceived average is a 7, then it doesn’t matter whether you give it a 4 or a 5. There’s no functional difference.

Since the 10 point scale sucks, we have to look at alternatives. Of the numbered options, here are the most common:

  • 2 point (up/down, yes/no)
  • 3 point (up/neutral/down)
  • 5 point (5 stars)
  • 100 point (percentage)
  • 1000 point (percentage with decimal)

I’m going to bin three of these right off the bat. 100 point and 1000 point have the same problems as a 10 point scale. If there’s nothing quantitative to differentiate between a 91 and a 92, why are you even using a 100 point scale? I’m also going to bin 2 point up/down scales too, for having the opposite problem of no nuance. Some things are just average, or they really work for a niche but no one else, and 2 point scales don’t allow for anything but good or bad. They work for probably 90% of scoring needs, but really don’t for the rest. Criticism demands a space for something other than worst or best.

This leaves three point and five point scales. Both scales escape the problem of the 7. Both scales have more nuance than up/down. Both scales have obvious differences between scores (unless you’re doing a 5 star scale and using half stars, then you’re just using a 10 point scale). I lean closer to the 5 point scale for preference, because those extra two points between best of the best and worst of the worst can save a lot of criticism from scoring everything as mediocre. But this leads to my next complaint about criticism.

No One Uses 1’s or 10’s and That’s Bullshit

Perfect and worst scores are rare on a 10 point scale. Why? Because they represent the absolute pinnacle and the absolute pit. When you have a 10 point scale, the temptation is always there to give a 9 instead of a 10 because 10 represents perfection. But nothing is perfect. 10 and 1 are essentially unattainable; reserved for the best of the best and the worst of the worst when everything is scored at a moment in time. Even if you do something like a look-back review, your score isn’t likely to increase over time. It’s more likely to decrease if anything, because you’ve got newer works to compare it to, or because it’s being looked at outside of its bubble in time.

By making 1’s and 10’s off-limits, reserved for exceptionally exceptional works, your 10 point scale becomes an 8 point scale. 2 and 9 become comfortable because they should be 1’s or 10’s but we can’t get over that need for perfection or utter failure. Now combine this with an assumed 7 average, and you end up with the IGN problem: everything great is an 8.

I’m 100% guilty here. I’ve given 9 scores to excellent games, games that should be a 10. I’ve also been reticent to give a 1 to anything I’ve finished because, hey, I finished it right? Even worse, when I’ve tried to translate 10 point scores to a 5 point system, I’ve found myself rounding down, so those 9’s became 4’s. I’ve given reviews to things I haven’t finished a 2/5 because… why? I didn’t finish it! Isn’t that a sign that something’s very wrong?

It takes some bravery to give something a 10 or a 1, but I want to see more bravery in criticism. I want to hold myself accountable to this. Great work deserves a 10. Garbage/DNF stuff deserves a 1. Mediocre things should be a 5, because a 7 average is wrong. And all scoring of media is bullshit, because it shortcuts the actual criticism and lacks nuance, but it can be better if we stop using a 10 point scale, and start using the far ends of whatever scale we’re working with.