Pimp my option choices!

In my mind one of the biggest turnoffs for respondents completing online surveys is the utterly uninspiring language we use to write them with.

I have talked in this blog about the impact that taking a more creative approach to how we ask questions in online surveys and how this can improve the quality of responses.

I would also like to challenge the way we think about using option ranges in surveys too. Particularly the over reliance on tired and unimaginative 5 point likert scales.

To me phrases "like how much do you agree or disagree with this statement” or”” how appealing or unappealing is this and “on a scale of 1 to 10” come across like clichés and do nothing to stimulate our imagination of respondents when they are answering questions, lack any real context, often don't fit the range requirements of the question.


These are the 5 questions you need to ask yourself if you want to design more effective range options:

1. What is the emotional impact of the range choice you are offering?

Take this very personal example.  I often ask my son on a scale of 1 to 10 how much did he enjoy his day as school.  The question does not inspire him to really think about the answer and so invariably I get either a 1 or a 5 and sometimes a 10 as the answer.  He gives me lip service to answering a question because he has heard several times before and so answers with a cliché  response.

Now one day I came home and asked on a scale of 1 to 10 how much did you enjoy your day at school but change the range choice.

10 = was the best day ever, they cancelled lessons and you all went to Disney world and had free ice creams 

1 = the worse day, all you did was Maths, the teacher kept you in at break times and for lunch all there was nothing to eat but cabbage.  

Now this was the first time I ever got a 6 out of him. 

By adding some emotional relevance or resonance to the option ranges it forced him to properly think and process the question.  This is what a good set of option ranges should do, they are not just there to benchmark, they have an important role at encouraging people to think.

2. What is the context?

Standard range choices can often be intangible choices for respondents to answer as they are so general and un-contextualized. When constructing the range choices you need to think about what actually you want to know.

Take the example the question how much do you like chocolate? Are you asking this in the context of other people or other things? My sister for example absolutely loves chocolate, a lot more than me, and I like chocolate certainly a great deal more than I like Brussels sprouts, but then again I would not like chocolate with my Christmas dinner and I probably don’t like chocolate as much as I like ice cream.

i.e. it's  really difficult to say how much you like something unless it is the context of something else.

Take another example the question “how much do you like watching golf on TV” the easy option would probably opt for a standard 5 point likert liking scale (like a lot, like a little, neither like/dislike, dislike a little, dislike a lot etc) but again with this you are offering no context a better solution might be…

How much do you like watching golf on TV?

I Would turn off the TV if that was all there was to watch

I would watch it if nothing else was on

I would watch in preference to some other programmes

I would watch in preference to most other programmes

I would be really annoyed if had to watch something else

Now I would not say this is a perfect set of answers either because for example your view point on golf may vary if it were say the Ryder cup or a local tournament, but I hope it illustrates my point that this is a range respondent I think I would find this easier to answer than a standard liking question and I think as a researcher you could argue that the answers you go from this would probably be of more value. 

Simply put the liking has been contextualised.

3. What are the natural range limits?

What researchers normally want is a good natural spread of opinion from a question to enable them to differentiate the answers. One of the other issues with offering a standard 5 point likert range question is that the options often do not naturally spread across the range, they tend to get clustered.

Think about the question “how much do you like chocolate?” Nearly everyone likes chocolate so to offer respondents a standard 5 point liker scales leaves the average respondent with only 2 choice which is a bit of a limited range, like a little and like a lot.

Or take the example of how films get rated.  The 1 to 5 star ratings are nearly all top weighted and clustered around 3 4 and 5 to the extent that if a film has a 3 star review we assume it is pretty poor.  Have you ever seen a 1 star review, they are very rare.

We are a bit stuck with the 5 star review process but it is not tremendously useful it really needs to be more exponential range set.


So it is important when thinking about a set of range choices to think about how the answers may get spread and adjust them accordingly.

4. Does the range cross cultures?

My 4^th point is that some standard ranges have a lot of cultural issues to contend with, for example the Asians prefer not to outwardly say they don’t like things so on 5 point range scales the answers get closeted around the top 3 choices, so this is a pretty useless question to use in these market.

5. What visuals can you use to support the options choices

The most effective means of  communicating and helping to emotionalise option choices is with the use of imagery.  Imagery really help respondents make choices more easily and are particularly useful if it is a repetitive question.   It is important that the images chosen accurately reflect the sentiment of each option choice, if they don't they it may be counter productive to use them but they really can add value if you are able to find a way of visualising a range option.   Here is an example below of how imagery can make answering a question a lot easier.

 Issue with adopting a more creative approach

You may instinctively worry about mucking around with using more emotional ranges, as to the impact it will have on the answers.

The answer will almost certainly be slightly different from what you would achieve with neutral/standard question ranges, but I would argue that in most case they will be better, richer, formed out of a greater amount of thinking.

Our opinions are 3 dimensional all an answer to a question is, is a 2d snapshot of that opinion and  a lot of standard likert scales over a very narrow'oblique perspectives on 3d opinions.

In my mind if by tweaking the question you do get a different answer this is an indication that the topic is more complex than you think and sticking to the standard constrained portfolio of question options will perhaps be a safe option for you but you may well end up with a very narrow perspective on a subject.  By switching to a different way of asking the question you can open up greater depth of thought and opinion and if it does throw up unusual answers, well this may be a sign that you need to do more research generally on the topic you are investigating.

Its not to say that you can pluck any whacky range set out of your head and think it will do, there is a lot of thought required.  The basic idea though is to humanize and use descriptive terms that trigger the imagination and contextualize the answers and you will get better data.