Is There Value in Surveying Your Customers Online?

There you are, minding your own business, buying something online, when a pop-up appears asking if you'd like to take part in a survey about your experience. There might have been a time when being asked for your opinion was in some measure gratifying, but for the most part, checking boxes on a form isn't a rewarding experience. It feels uselessly indirect if you have a specific grievance and pointlessly congratulatory if you’ve just purchased something.

For the companies using them, these surveys seem great. They provide them with diagnostic data at virtually zero cost.

However, there's now considerable scientific evidence to show that such surveys aren't capable of providing reliable data. Their value needs to be considered in terms of the impact they have on the customer experience (usually, but not always, negative) and an organization's decision making (which if it embraces the data as fact is corrupted).

At the heart of the issue is the unconscious mind. We rely on it far more than we’d ever like to believe and, with no direct access to its workings from our conscious mind, we're poorly placed to represent our thoughts in surveys.

Understanding how the unconscious mind shapes consumer behavior is both fascinating and, initially, a little unsettling. We're extremely susceptible to influences outside of our awareness. It’s the reason that moving a logo a few inches on a website can increase sales, or reducing the number of choices offered will often increase conversion. It explains why providing more search engine results per page can result in traffic declining and how reducing a map’s size by 25 percent can cause traffic to increase by a similar amount.

None of these changes are ones that people would anticipate or advocate in a conscious market research process. In fact, it's quite the reverse. It was market research that prompted Google to increase the number of search results per page. Fortunately the search engine was tracking its impact closely.

Psychologists have shown that simply asking someone what they like and dislike about a product before rating it changes their preferences across products. The question order, choice of answers presented and the way in which options are described have a huge impact on response. In one independently conducted survey by two companies that I analyzed, the difference was 60 percent.

Of course the vast majority of online retailers track their website analytical data very closely. The problem comes when such real-world data is expected to share a stage in the boardroom with the data from surveys and focus groups. The two sources appear very similar in a report, and the statistics have a solidity to them that feels tangible. However, the psychological validity of each is worlds apart. And whereas one is captured without disturbing your customers, the other in all likelihood does nothing to enhance the appeal of visiting your site again.

Philip Graves is the author of "Consumer.ology: The Market Research Myth, the Truth about Consumers, and the Psychology of Shopping." Philip can be reached at philip@philipgraves.net.