It’s time to kick the rating question habit

This is my first data science blog for a few months as I’ve been busy with the rest of our team preparing for the release of Audiem 2.0. So, what prompted me to put pen to paper again at such a busy time?

Well, it links to one of my previous blog posts about survey design. But this time I want to focus on a specific issue: the excessive and unnecessary use of ‘tick-box’ rating scale questions in employee surveys.

You know the type of question I’m talking about. Respondents to a survey are asked to rate their agreement with or the importance of a series of statements by selecting a point on a scale that best represents their opinion.

‍

It’s easy to see why rating questions have been so widely used over the years. They are structured and generate numeric data that are easy to analyse and digest, making them appealing to organisations looking for metrics on which to base decisions. But while their popularity is understandable, there’s a tendency to overuse them.

Earlier this week, I was looking at an employee experience survey and was struck by the fact that it had 84 (yes, eighty-four!) rating scale questions. And today I encountered another with 73 rating scale questions, out of a total of 94, the rest being a mix of categorical and free text questions.

I’ve seen many surveys like these over the years, but these examples felt particularly jarring because there are now far more effective and engaging ways to understand how people feel about their workplace experience.

I’m not suggesting that rating questions don’t have a place in surveys – they do and I’ve used them in many surveys over the years. But when they dominate a survey to this extent, they risk turning what should be a thoughtful feedback process into a mechanical box-ticking exercise.

Worse still, rating questions fail to capture the nuance and context behind people’s experiences. In an age where we have access to sophisticated tools for analysing free text responses, relying so heavily on rating scales feels like clinging to an old habit that no longer serves us — or the people we’re trying to listen to.

Amongst our clients, I’ve noticed that once they become more confident with free text feedback — and see how it can unlock richer, more actionable insights — their reliance on rating questions drops significantly. Their surveys get shorter, more focused, and far more engaging for the people completing them.

But at an industry level, old habits take time to change. Many organisations are stuck in a cycle of defaulting to rating scales simply because “that’s how we’ve always done it.” It’s time to challenge that mindset. With the tools now available to us, we can design feedback experiences that are smarter, more human, and ultimately more useful.

If you’re looking to reduce your reliance on rating questions and explore better ways to understand what people really think and feel, drop us a line at hello@audiem.io — we’d love to help.