Where can we inject more creativity into survey design

Here are my thoughts on some examples of the areas where I feel we need to inject a bit more creativity into the design of surveys. This content is taken from one of my presentations on the topic.

1. Think about how we motivate respondents to answer questions and take part in surveys

We so often jump straight into surveys with questions that look like this...

In other words, we jump straight into the nitty gritty of things we want to know.

I ask, has anyone in market research ever heard of foreplay?

If you want to encourage people to think, which is essentially the task we are asking respondents to do, it really does help if you warm them up and provoke their curiosity and interest in a topic first, before wading in with the boring questions.

There are lots of ways this can be done but one of simplest techniques is to try to think of a question respondents might actually want to answer and use it as your opening gambit.

Say for example you were doing a survey about toothpaste. You could jump straight in with questions about what brands of toothpaste they are aware of, or you could ask questions like which of these celebrities has the nicest teeth? 

i.e. You get them thinking about the topic in a more rounded way. We have found that this can stimulate them to take a lot more interest in answering the subsequent questions.

This approach to considering respondents’ motivation is at the heart of effective survey design. You should be thinking about this with every question you write in a survey.

2. Improve skills at copy writing

I think we need to make a fresh start in the way we think about writing questions in online surveys. We are locked into dry research speak that doesn’t connect with respondents. We are obsessed with the platitudes of 5 point, 7 point, 9 point scales and concepts like ‘strongly agree’ or ‘disagree’, which are often meaningless and ineffective in emotionally engaging respondents. Go out and listen to consumers describing brands, and I don't think you ever will hear the words ‘very appealing’ or ‘I strongly like this brand’.

If you want to connect with respondents, it helps if you use their language. Tell me, what does ‘strongly agree’ actually mean to most people? You either agree or disagree with something.

We have a general over reliance on range scales, which to me is just a lazy approach to research. Respondents in turn are extremely lazy in the way they answer them. If you roll up the answers to a cross section of say 10 point range scale questions, you will find that nearly 80% of answers are clustered between 5 and 8. Respondents are not being encouraged to think. Scales mean nothing to most people, or rather, we have learnt to use them in a certain way.

Sweep away verbosity

We need to clear away the verbosity of instructions which may be essential in a face to face interview, but are not needed in questions which have visual cues. I would recommend applying a twitter rule of no more than 140 characters for a question. Believe me, no matter how important you think the instructions are, a huge proportion of respondents do not read past the first sentence.*

* roughly speaking 50%!

Note: if you do have a more complex idea to communicate in a survey, then this needs to be broken down into clear parts and presented presentation style, one thought at a time. Injecting imagery will be a powerful tool to help communicate the message.

Write questions with a sense of humanity

We are habitually autistically precise in our phrasing, which can be completely alienating to respondents. I would recommend writing from the point of view of an advertising agency trying to sell the question to the respondent, or a TV interviewer questioning a famous celebrity. That’s not to recommend adopting the vernacular or colloquial, but question wording needs to be clear. Try to use what might be described as ‘natural language’.

Think more conceptually about how to ask questions

And realise it’s not just about the wording, it’s about the underlying concept. For example, you could ask people to rank their first, second and third choices of toothpaste, or you could say to respondents: imagine you are a judge in an award ceremony and you have to give out 1st, 2nd and 3rd prizes in the Toothpaste of the Year competition. A conceptual shift of emphasis like this that can make surveys far more interesting for respondents to answer. We too often rely on a cliched armoury of standard questions and approaches to problems, and these are habits we need to break!

3. Storyboard the flow of surveys

My next point is the lack of storyboarding in surveys. We throw respondents a jumbled mess of questions, which for the respondent has no sense of flow or structure. We ask them to think of one topic, then another, then the first again. We deliver these horrible never-ending loops of questions that make the respondent give up all hope that they will get to the end of a survey. I feel that much more thought is needed in the way we organise and deliver questions and signpost their order so respondents can grasp where they are and where they are going. It may not be important to you; it is to respondents.

4. Ask fewer dumb questions

All the research we have conducted over the last few years into what makes surveys interesting to respondents leads to one clear observation: respondents like nothing more than to actually think! Yet so often we bombard them with dumb questions that don't require more than momentary thought: how much do you agree with this, do you like this, or pick some words. Very rarely do we actually ask them to think. I encourage you to think of respondents as consultants; try to bring them into the problems you are trying to solve. Ask them more intelligent questions and I think you will be surprised at the quality of feedback you will receive. 

5. Up our game on the visual side

I don't think many people have grasped the impact that quality imagery can have at improving the experience of taking a survey, how well-chosen images can be used to communicate ideas more efficiently or the impact that an image can have on stimulating the imagination. I very rarely see imagery properly used in surveys.

More often what I see is badly chosen clip art slapped to the top of a range question, and imagery that looks like something the intern has been tasked with finding.

6. Rethink the use of likert scale grid questions

I think the industry needs to have a total rethink on the use of likert scale grid questions in surveys. They are simply not working. Respondents hate them and then deliver, at best, watered down data, and at worst, heavily corrupted data.  They are officially my number 1 bugbear with surveys.

If you take out of the equation reading times the average respondent spends 4.3 seconds thinking about the answer to an average question, but as low as 1.7 seconds considering the answer to a likert scale question in a bank of grids. If you look in detail at the data coming back from a grid question, upwards of 80% of respondents show some signs of speeding after the 20th repetition, and I see surveys with repetitions like this in their hundreds.

Part of the problem is our obsession with agreement and liking and 5 point scoring scales. Respondents are so tired of these we have an almost pavlovian response to answering them - we mostly say that we slightly agree, slightly like everything!

What we see if you offer up more creative scales can be radical improvements in attention. To my mind an agree scale is just a really lazy way of asking a question.

How much do you like watching these sports on TV on a scale of 1 to 5? Every sport will score between 2.5 and 3.5! Surely the point of most questions like this is to draw out comparisons. Just look at the behavioural reality of our attitude towards watching different sports on TV; Football would score 100 and most other sports under 10.

The solution is to think about how you ask the question more creatively, understanding the real objective of the question, using more relevant anchor points for these types of judgements and using more bespoke ranges for each measure.