A few years ago part of my organisation conducted a big (and no doubt very expensive) survey. From what I can recall, it was aimed at gaining Insight into the UK population’s use of technology and the media and what sorts of individuals and households they were.

The results were published in a blaze of publicity and I have to say it was all very interesting. But some of the findings were surprising (to me at any rate). It seemed to be portraying an incredibly positive and techno-savvy view of society that didn’t quite tally with my experiences: most people were using the Internet for everything, and those that weren’t soon would be. Furthermore most people had the latest gadgets, and those that didn’t indicated they soon would have, and people were drifting away from ‘traditional’ technologies and ‘traditional’ media and moving on to new alternatives – inevitably on demand and internet based. And you guessed it: those that weren’t, indicated they soon would be.

Here are a few more findings:

  • 97% Are concerned about the impact their purchasing decisions have on the rest of the world
  • 84% Are willing to volunteer their time to good causes
  • 78% Are optimisitic for the future
  • 77% Like to get involved in helping out the local community
  • 60% Have shopped online in the last 12 months

Whilst this is encouraging, it all seems a bit too utopian to me. Too perfect. Almost contrived, somehow.

I am not for one moment disputing the accuracy or presentation of the data collected. I have no doubt this reflected what people genuinely said. Also, I think surveys can be useful.

But surveys have a simple and quite glaring deficiency that seems to get overlooked. That is that you can’t force people to participate in them. If I go out on the street and ask people to complete my survey about how much they like the Harry Potter books, I will quite likely return with a very high “yes I like them” result. But this is because the people that don’t like Harry Potter are unlikely to participate in my survey. I will likely only get the ‘like them’ population participating. Furthermore you have to have a certain mindset to participate in surveys at all- many people avoid them at all costs or out of principle. Regardless of the subject matter. And why not? That’s their right.

But it means the results are skewed right from the start.

Participating in a survey and saying ‘no’ isn’t the same as not participating at all. Further, the people that don’t participate probably have something interesting and enlightening to say, if only we can get them to tell us. But how? To go back to our Harry Potter example, it would be all to easy to extrapolate my results to mean “most people like Harry Potter”. Well, how do we know? We don’t.

So I have nothing against surveys. The problem I have is that people all too often try to use them to make ‘fact’ and empiracle ‘truth’ where there is none. Surveys are just a tool in their delusion. The world is still a complicated place, and it is what people don’t say that is often the most interesting.

« »