Big data and the home chemistry set

Are we all Dodos?   I heard a couple of people tell us at the ESOMAR 3D conference that we are perilously close to extinction,  that we market researchers are dodos. In fact this has been a bit of a common theme at many of conference I have attended in the last few years a prediction of the terminal decline of research as we know it. The message is that our industry is gonna be hit by a bus with the growth of social media and the big boys like Google and Facebook and IBM muscling in to our space. We are also in many parts of the world facing tough economic times and tightening budget.

Yet despite all this it appeared that this was the best attended 3d conference ever, and it's not just this isolated conference either. I have been going to research conferences all around the world over the last year and they all seem to be seeing growing numbers of attendees and all I can sense from these conferences and particularly at this event, is an industry brimming with confidence and ideas.

So are we all putting on a brave face? Are we naively sleep walking into the future?   I don't think so...


The macro-economics of market research

Over the last decade we have seen a near exponential growth of data being generated by the worlds population. It is literally pouring out of the internet and our mobile phones.   We also have an ever increasing range of innovative ways to measure and analyse things, ranging from geo-location tracking right through to sensors attached to our heads.  We are able to measure almost everything we do. So who is going to do it? 

 We learnt at this conference how hopeless computers are at actually thinking and they lack the ability to really intelligently analyse data to the quality and standard needed to glean real market research insights. In every presentation we saw the critical contribution that human being had.  It just leads me to believe that with all this expansion of data the pool and means of measuring things more people are going to be needed to make sense of it all and who is best at doing this?  Ultimately I believe it is market researchers.  Media companies and consultancy firms might think they are in with a shout but I genuinely don't believe they have to core competences needed. Market research is all about working out how to measure things, gathering and analysis of data and using this to delivering insights. That is what we specialise in, that is what we are expert at.   No other industry is better placed to capitalise on all this free flowing data than ours.

The big thing I think we need to focus on is developing new tools to process big data. Right now it looks like we have oil tankers full of information waiting on our door step and the research industry is currently attempting to use tools that look like they are from a home chemistry set to try and process it into fuel.  

We need to develop more tools to refine all this information and learn the skills to do it.  That is our challenge , grasping the technology that is out there and adapting it for our needs.

But I have every confidence we can. I believe we are the best industry out there at cross communicating ideas and with that comes innovation. There are lots of people who bemoan the lack of innovation in our industry and again I see quite the opposite. I see an industry racing to innovate.  Step back and look at how things have developed in the last couple of years alone.  Look what we can do already in the field of mobile  research,  how many research companies have so quickly moved into offering social media research solutions. Look at some of the fantastic tools that have emerged like facial emotion measurement and neuro/biometric monitoring, look at what we are learning and embracing from the fields of behavioural economics, look at crowd sourcing and the success of MROC communities and how we have developed technology to serve these communities. Look at some of the new text analytics tools that are emerging.

I think the market research industry is more than capable to adapt to these needs and I feel it has a big future.