200 question surveys?!!!!

In the presentation I gave yesterday at the MRS conference I mentioned that we had been working on some experimental survey games where we had managed to get people to voluntarily complete a 200 frame survey.

Now firstly, this has been quoted as a 200 "question" survey which I am afraid is a bit of an exaggeration, as a lot of the frames were feedback pages and not questions, it was about 120 questions in total.  I apologise I did not make this clear during my presentation.

I am NOT, may I repeat, NOT espousing or suggesting that anyone does a 200 question survey or indeed a 120 question survey for that matter!!!!!!!

I am slightly concerned about the mixed message this is conveying  and so I thought I should clarify things a little with this blog post about the details of this experiment.

This survey in question was designed purely as an experiment to see how many question respondents were prepared to answer, when instead of doing a survey they were playing a game and getting feedback that was  of some use and interest to them. This was not a traditional survey but a shopping game we had specially designed, that stepped away from the thinking constraints of a typical survey and focusing purely on the game and feedback mechanic.

The respondents had to work their way through a series of  "levels" where they were are asked to do things like guess the most and least expensive products, the most and least popular products, the prices of products and try and predict what different celebrities and types of people would buy. See below screen grabs of what the survey looked like.

 They would get points for getting things right and at the end of each level they would find out how well they did and we also revealed to them what this told us about the type of shopper they were.  So for example the respondents found out how price concious they were compared to other people and whether they were a social shopper who buys popular products or a individualist who buys not so popular products.  Each level was voluntary, they were asked if they wanted to proceed to the next level. There were 6 levels in total and 15-20 challenges in each level and we found 94% voluntarily completed all 6 levels spending over 20 minutes on average completing it.   The survey had an enjoyment score of 9.0 out of 10. The highest audience evaluation score we have ever achieved for a survey.

We often say that surveys should not being longer than 20 minutes. That is because most if not all surveys are not entertaining enough to persuade  us to want do them for any longer. Most surveys fail to cross the entertainment divide.  20 minutes is in effect a tolerance limit for expecting anyone to do anything they find boring.  But if you start to look at a survey through the lens of being a piece of entertainment or a game then yes it does open up possibilities for surveys that are genuinely entertaining to be longer than 20 minutes.  After all we happily will watch a film for a couple of hours, read a book all day on holiday play Angry Birds in any spare waking moment we get with our mobile phone.   But the aim of this research was not to path the way for, or encourage the industry to start churning out ever longer dull surveys!