Game Theory – turning surveys into games

Gamification is clearly a bit of a buzz word in the marketing industry right now and a subject of growing interest amongst market researcher as witness by the enthusiasm this topic generated at the recent NewMR festival web conference organised by Ray Poynter that I was part of.

This is an article I wrote following on from this conference for MrWeb that I am reproducing  that looks at the work we have been doing to explore the role of game play in online surveys.

Background

For the last 3 years we have been on a quest to find ways to improve online surveys by making them more engaging for respondent, conducting over 100 experiments looking at different techniques. Over the course of this work, we observed how impactful introducing any level of playfulness or game-style activity in an online survey could be at stimulating extra feedback.

I would go as far as saying that you could single out game play as the single most effective means of engaging with respondents we have discovered.

As a result, we began last year to take a serious look at how we could introduce more game-like activities into surveys and researching the impact this had. 

For those that are not already tuned in to survey Gamification techniques, here is a short summary of what we have learnt so far.

1. Reframing questions to be more game-like:

A game is really anything we do that involves thinking that is fun. What differentiates a game from a survey question really boils down to how we ask it.  Take this example:

Question:  “what is your favourite meal”

Game:   “Imagine you are on death row, what would you order for your last meal” 

By asking the question in a more imaginary framework it becomes more fun for respondents to answer and in turn we have found respondents then deliver far richer responses.  (see examples below)

I was recently interviewed by Surinder Siama from Researchtalk and we talked about this technique and he challenged me to put my money where my mouth was and asked me to do a live experiment with a couple of randomly selected people in the office, he recorded it and here are the results:

http://www.youtube.com/user/researchtalk#p/u/0/MybClB6cAQ4

2. Adding a competitive element

Most games have a competitive element and it is amazing the impact that adding any form of competitive activity to a survey can have.

The simple phrase “We challenge you...” added to a question asking people to recall something can easily double the volume of responses.  Other phrases like “can you guess...” or “you have 1 minute...”  seem to be equally powerful.

Question:  “what brands do you recall”

Game:   “can you guess the top 5 brands, you have 2 minutes...”

This shift in emphasis not only improves response, in this above case delivers 4 times as many brand names we see how it also improve enjoyment levels, highlighting the direct relationship between fun and effort.

3. Reward and feedback mechanisms

A typical survey contains no form of feedback or reward for successful performance, except a somewhat distant financial incentive.  Yet you only have to look at the success of at psychometric test style games flying around Facebook to understand the power of a reward/feedback mechanic.

Question: “what is your favourite colour”

Game: “Find out what your favourite colour says about you!”

Reward and feedback mechanisms are extremely easy conceptually to integrate into surveys and we are working several clients developing these right now.  The issue really is in adapting your survey technology to handle scoring and feedback mechanics effectively. This is something we have been focusing on from a technical point of view and now offer a scoring component that can ne integrated into any survey.

4. Understanding that we enjoy thinking

Many of the games we most enjoy involve often involved quite complex thinking, take chess or Scrabble or most computer games as examples. Yet many surveys simply do not have any expectations for people to think at all,  Conjoint research is a perfect example of this, a mind numbingly dull task for respondent.  We have played around with transposing traditional conjoint research to more complex product building games and the impact is dramatic, delivering far more creative solutions to the extent that we are now building a full product offering using this particular game play technique called “Evolution: survival of the fittest product”.

Question:  “would you prefer a 10’ thick base pizza, or 12’ thin base”

Game: “Who can make the most popular pizza from these ingredients?” 

5. We play games with other people

Playing within a group, or as part of a team, brings the potential for collective spirit, which can engage greater effort and increase desire to complete a task effectively.

Clearly, this is much harder to realize in an online environment as we complete a survey as individual. 

But not impossible, the idea we are exploring right now, is to create a series of surveys and allocate respondents into teams and they compete against each other to complete a series of task. We are doing some experiments to see how this works, but the technique of using team based game play I already know is being employed extremely successfully in focus group by Arthur.Fletcher & Blauw Research.

Question: “what is your solution?”

Game:  “Can your team come up with the best solution?” 

6. “Gamifying” questions

Working from these initial ideas, we have explored how to redesign conventional question formats, such as text input, word selectors and grid questions, to make them more playful and game-like in character and are building a suite of more game style question format into our survey technology.

Question

Game

Some of these new question techniques are still under wraps while we are testing them out, but hope to soon understand the impact that they can have on how respondents answer.

7. Turning whole surveys into games

The final thoughts is on more holistic approaches to game play.  I think the field is open here to a wide range of ideas. Games can take you into all sorts of mental spaces and do a lot more than just engage.

The most successful approach we have found of doing this so far has been to employ projection techniques, such as asking respondents to imagine they are the boss of a company.

Question:  “what do you think of this new product idea”

Game:  “Imagine you are the judge on a new TV game show called new product factors”

This technique we have found can result in 3 times as many thoughts and ideas from respondent but I would also at a lot more freeform candid and creative feedback.  

In summary

The basic idea behind game theory as applied to market research is that respondents who perceive a survey as an enjoyable game-like activity are much more likely to devote effort and thought to its completion, and thus give more valuable answers.  In game play more respondents are prepared to do more adventurous things, are more creative in thinking and you have the opportunity to tease out feedback from people you could not get in conventional surveying.

Game thinking is really as much of a mindset about engaging respondents and challenging them to more adventurous as it about the specifics of the methodology or about designing wacky questions. 

I believe that nearly any question in a survey could be made more fun and game like to answer with enough thought and imagination and I predict we will see whole new game style survey mechanics emerging in the future.


I appreciate as a survey design company we focus on the mechanic of engagement but I think game play has a lot more to offer that just this.  They have the ability to take people into different mental spaces and mind sets and who know really how this could be exploited. 

 

We are currently working with Engage research and three end-user clients to explore ways of making whole surveys more game-like and we are also looking at how these gaming technique impact on the character of answers, an important question for many people. We hope to reveal the results of all this research in a paper we are intending to publish later this year.

Updates:

Here is a feature article on survey gamification from Research Magazine I contributed to:
http://www.research-live.com/features/an-extra-life-for-online-surveys/4005075.article

This is a link to a recording of a presentation on gamifying survey I gave at the NewMR festival event in Jan 2011 which has just been made publically available: 

http://newmr.org/page/gaming-and-mr#jonpuleston

Upcoming presentation on Gamification of surveys:

I will be presenting a full paper on this topic with Deborah Sleep from Engage Research the ESOMAR congress in 21st Sept:
http://www.esomar.org/index.php/events-congress-2011-presentation-abstracts.html#the-game-experiments

And a more technical paper on survey gaming techniques at the the ASC Conference on 22nd September:
http://www.asc.org.uk/events/september-2011

I will also be presenting on survey gaming at the BAQMAR annual conference in December:
http://www.baqmar.be/?page_id=343