Reads Like A Seven
The other day I had a conversation about review scores and found myself thinking back to when I used to write about videogames and the seemingly endless arguments over whether reviews should carry ratings (and all the many fun sub-arguments about what rating system they should use). Some of you will remember the good old days of Gamespot reviews splitting out scores for things like "graphics" and "fun factor", the frankly bewildering 100-point scales of PC Gamer, Polygon not having scores for reasons of high-minded principle and then realising actually maybe they were fine if it meant they'd be listed on Metacritic, etc. I had to explain to a friend without any context for videogames media the classic "7-10" scale[1] and they were utterly befuddled.
Despite having left games reviewing behind a long time ago, I feel I am being asked to rate things more and more lately. Delivery companies, to take one example, will send you emails asking you to "rate your delivery" out of five—I have a screenshot of just one such here:
What do the people sending these think is a three-star delivery? A satisfactory delivery is pretty much a binary state: there's one "success" outcome (item delivered to me intact and on time), and while there are a few ways it can go wrong (delivered late, to the wrong place, or damaged) these can all pretty easily be collapsed into "failure". There isn't really anything that can be done to make it better if you've given me the parcel intact, unless the delivery driver were to give me some home-made cookies along with my package[^2].
There's a bit more gradation in terms of what a "good" or "bad" taxi ride might be, but it's been my experience that everyone gives the Uber driver five stars (unless they drove like a maniac, spent the whole time subjecting you to their novel theories on race over a soundtrack of thumping techno and still got you there late) because you know that functionally the system is either five stars or nothing, and they'll be punished by the app for anything less.
There's a hedge fund called Bridgewater Associates which is somewhat notorious for having a system where all its employees rate each other all the time. Sometimes I feel like the whole world is becoming an even more foolish version of Bridgewater. When you ask someone to rate something complex on a scale without rigorously defining the scale, the answers you get back should be considered anything from "a response to the obvious incentives of the system" to "pretty much totally arbitrary".
fun to mock though it was, I'm inclined to agree with whoever observed that because big games take a long, expensive time to develop and very often cost just as much to market, for the most part if they fail to clear the low bar of basic competence they just don't get published. ↩︎
- ← Previous
Fresh start - Next →
Don't Forget We're Here Forever