I do a lot of online consumer and marketing surveys – it doesn’t pay much, but every little helps (most of the cost of the new mattress we bought last month came out of my survey money).
Having been doing them for years, I’ve come across far too many badly designed surveys that don’t ask the right questions and don’t give enough options. For instance, most consumer surveys assume that everybody does all of their food shopping in one of the big supermarkets, never cooks from scratch, never brew their own wine and beer, and never grow their own fruit and vegetables. So I find quite a lot of surveys about shopping and food quite hard to fill in accurately; lists of supermarkets only occasionally include the Co-op, corner shops rarely feature, and as for street markets and farmers markets….
One particularly clumsily-designed survey was asking questions about furniture-buying; the first question was “Have you bought an item of furniture in the last 6 months?”, to which I replied “A kitchen table”. Then I was asked which shop I had bought the item from and presented with a long list of furniture retailers. But I hadn’t got it from a shop – we had bought it from one of our neigbours, and there was no option to say so. There was only a box at the end of the list, marked “Other”. So I entered “Our neighbours” in that. As a result, I had to answer a further twenty or thirty questions on the “Our neighbours” furniture shop – was their shop well laid out, were the staff helpful, was I offered different payment options, did they have a catalogue, did I go to their website etc. etc. Not a complete waste of my time – I got 50p for it – but certainly a waste of the marketers’ time.
Another useless survey I remember was one on energy utility companies. First of all, I was presented with a long list of energy companies, of which I had only heard of around half and of which I only had direct experience of two. But I was expected to give my opinions and impressions on each and every one of them. I did try, and might have succeeded, except for the bizarre jargon the questions were couched in – they sounded like they had been written by a 20-year old London-based marketing graduate who perhaps did not have English as his first language. Some of them were simply impenetrable; they may well have made sense to marketing people, but certainly not to me. The one that made me finally give up on the whole thing asked me to choose an energy company that fitted the statement “This company speaks my language”. Since there was no option for “Whichever one speaks English and not bollocks”, there was no way I could complete it.
Another slightly bizarre survey came in this morning. It was about bread, concentrating on a particular bread brand. During the course of it, I was presented with five groups of photographs; the photos were all of a variety of people, showing only their faces. Presumably, these five groups were supposed to represent different demographic groups, although the only real differences between the groups that I could see were the different colours of the backgrounds. The questions at first were mainly about my feeling and opinions on which type of bread would be favoured by which group, and which statements about the bread they might make. I’ve had this same sort of survey a few times before, and I never know how to answer it – just how am I supposed to choose what bread a bunch of imaginary people might like, what they might feel and say about a product, when I’m told nothing at all about them? The idea is obviously to go purely on the look of the people in the photos, but what selection criteria am I supposed to be using? As usual, there was no guidance, so, as usual, I clicked on options more or less at random, going by the background colours of the photos more than anything else – feeling in a brown mood today, I chose the ones with the nice dark brown backgrounds.
But it got worse. I was then shown a TV ad for the bread brand. And I was then asked again to choose one group of photographs. And then I was asked “How would these people describe what the ad for Brand X bread was saying about the type of people who would like Brand X bread? ” and “How would these people describe what the ad for Brand X bread was saying about the feeling Brand X bread gives them? ” (These are the exact questions – I copied them so as not to lose them for posterity.)
At that, I could only sit and stare at the screen, trying to process these questions- it may as well have been a piece of computer code. Eventually, I decided to treat it like computer code and worked it out by going through it word by word – I was being asked to put myself inside the minds of a group of imaginary people (the photo-people), about whom I knew nothing, and:
(Q1) articulate their opinions and impressions of the audience the TV ad was aimed at;
(Q2) say what they (the imaginary photo-people) perceived what the ad was saying about how they (the imaginary photo-people) felt about the bread .
It was all way too metaphysical and circular for me. Q2 especially would made have made a good discussion topic for a philosophy group (perhaps after everyone had familiarised themselves with Plato’s Cave), but I’d only signed up for a marketing survey. Stabbing randomly at the answers, I got to the end of the survey as quickly as possible. Then went and sat down with a strong cup of tea and a soothing killer sudoku puzzle – so much simpler!