Making Statistics More Significant

Are individual conversations statistically significant for customer research?

No. By definition, one-on-one dialogues will not uncover data that applies to all of your clients. However, they can provide essential insights into how to craft your survey questions.

In The Hitchhiker’s Guide to the Galaxy, Douglas Adams writes of a supercomputer tasked with answering The Ultimate Question of Life, the Universe, and Everything. Finally, it reveals the solution: 42.

"Forty-two!" yelled Loonquawl. "Is that all you've got to show for seven and a half million year's work?" “I checked it very thoroughly,” said the computer, “and that quite definitely is the answer. I think the problem, to be quite honest with you, is that you’ve never actually known what the question is.”

Imagine that you’re a product manager at a software company with 1,000 clients. You want to poll all of your clients about what feature to build next. If you just guess what potential features to list on the survey, you may leave out the features that customers really want.

The solution: talk to a few customers first. They will give you testable hypotheses about what features to add to your poll. You may find that a huge swath of customers want a feature you would have never thought of on your own.

Of course, you could simply leave a blank space on the form for customers to suggest their own desired feature. But without guidance, your customers may not really know what they want or need. An authentic conversation can uncover not just perceived pain points, but the core issues that are harder to articulate.

Even clients who know what they want may not take the time to craft thoughtful responses on their own. After all - how much thought do you put into the company surveys that you receive?

Just as Picasso called computers useless because they “only give you answers,” survey are only as good as the questions they start with. If you’re not using real client feedback to build your surveys, you may find that your statistics aren't all that significant.