Performance appraisals are a common feature of the modern workforce, and they’re designed in part to provide objective feedback about an individual.  Of course, often they’re anything but objective, as the biases of the rater tend to distort their perspective on the person they’re rating.  A recent study from the Rotman School of Management highlights how it’s not just our personal foibles that open the process up to bias, but also the way ratings are structured.

The study explored whether something as straightforward as shifting from a 10-point rating scale and a six-point rating scale would make any difference to the kind of ratings offered, and indeed whether the ratings would change noticeably for different groups.

Interestingly, that is exactly what transpired, with the substantial gender gap evident in the 10-point system practically disappearing when people were rated on a six-point scale.

Academic ratings

The researchers tested a shift from a 10-point to a six-point rating scale in a large North American university, with students given the chance to rate their professors on both scales.  The school has a minority of female professors, with as few as 1 in 5 in around half of the subject areas.

Previously in the 10-point system, male professors would receive a 10* rating over 30% of the time, but female instructors only managed to do so 19.5% of the time.  This changed entirely when students were presented with the six-point scale however, as then not only did male professors achieve perfect scores 41.2% of the time, but their female peers did so 41.7% of the time.

These findings were replicated in a second experiment where students were presented with a transcript of a lecture before being asked to rate that lecture.  The lecture was modified so that students thought the lecture was delivered by either a male or female professor, and the two different rating scales were used in equal measure.  Similar results were achieved.

Gender bias

Whilst the authors caution that such an approach won’t remove gender bias entirely, it can help to limit the expression of those biases somewhat.

“Gender biases are particularly strong when people are making very fine distinctions between brilliant versus very good performance,” the authors say. “With a six-point scale, we are basically taking away the ability to make those fine distinctions.”

It stands to reason that the whole point of performance management systems is to provide a fair and unbiased platform for gauging performance, so any tweaks to the design of these platforms to ensure that happens has to be applauded.  Whilst it’s perhaps fair to say that more work is required to show that the changes to the scale used in this research are repeatable in a variety of contexts, it does nonetheless provide some interesting food for thought.