Leave No Stone Unturned: How to write research questions
Writing research questions that can get us the necessary answers is vital. And it can turn into a question of company survival.
“The cost of getting it wrong is so destructive,” said Rob Fitzpatrick, author of “The Mom Test.” “My first company, we must have lost three or four years because we asked dumb questions. After a while, that gets expensive.”
Asking the right questions can help us avoid wasting money, time, and effort.
In this article, I discuss the following:
- Why does the phrasing of research questions matter?
- What makes lousy research questions?
- How to write research questions that get us good answers
- The results of good research questions
Why does the phrasing of research questions matter?
At the core, it comes down to the fact that humans oftentimes try to please. So if somebody asks, “do you like my product?” that person might respond with a “yes” just because they wanted to please the person asking.
“A good way to get around that is to ask them about a competitor or a thing that may be similar,” said Jamin Brazil, managing director at Voxpopme.
And sometimes, consumers can’t even picture what they are being asked about because the product idea isn’t a thing yet. Jamin shared the example of research around web TV years ago.
“This was a new concept, and there wasn’t a lot of connection to it,” Jamin said. “But if you ask people ‘what do you think about always-on TV through the internet?’ who is not going to be like ‘yes, let’s do that.'”
But the product wasn’t performing that well, so researchers changed the questions to get to the bottom of things, including:
- What was the last piece of technology that you bought?
- Why did you buy it?
- What problem was it solving?
“You can start understanding purchase behaviors based on similar products,” Jamin said.
What makes bad research questions?
Questions that ask customers to anticipate their future behaviors can be less than helpful, Rob said.
“If you are trying to build something new, those traditional ‘what do you want?’ break down,” he said. “People are really bad at predicting their future needs and behaviors.”
Sometimes, consumers might inadvertently try to support the person asking the questions. For example, let’s say you are thinking about creating a new product and run it by several consumers.
They may say, ‘oh yes, that sounds like a great idea.’ And then you create the thing, and nobody buys it.
“Then they’ll say, ‘oh, we don’t really need it,'” Rob said. “But you told me it would be so innovative? That you loved it. ‘Oh, it is innovative. We just don’t need it.'”
In addition to the actual content of questions, context is essential. For example, let’s say we ask consumers if they are interested in this product that will help them be happier and healthier, and they might even get abs. They might even live longer.
“And they’ll go ‘yes, I want that,’ and then you go it’s called the gym,” Rob said. “And they go, ‘no, I don’t want that.'”
The positive, out-of-context reaction happens because they were sold the benefits without all the other things they would have to consider – including finding the time, parking, etc.
“In many cases, you can find parallels for lots of different categories and products,” Rob said.
How to write good research questions that get us good answers
Before writing the question, be clear about what you are trying to get out of the question. What answer will be helpful, said Brian Monschein, Voxpopme’s vice president of research. Once you start writing the questions, get to the point.
“Remember that there’s another human being on the other side of that survey, and they will lose interest,” Brian said. “So be thoughtful of how much you want to ask them.”
“A good rule of thumb: Take the survey yourself, as if you were a respondent, and see what you think,” Brian said. “If you struggle in any areas or lose interest, chances are your respondents will, too.”
Write conversational questions, and avoid robotic language or jargon.
“And don’t lead the witness, “Brian said. “Instead of saying ‘did you have a positive visit,’ ask ‘how would you rate your visit?'”
How to ask good questions
A good question is designed to prevent respondents from lying to you – even when the lying often isn’t will intent. Rob calls that the “The Mom Test.”
“If my mom is the most supportive, most biased person, I can structure the questions in a way that even she can’t lie to me, then it’s probably a good question,” Rob said.
An example
Don’t ask specific future product questions:
“Hey, Mom, what do you think of my yoga training at home?” or “What do you think of my awesome iPhone cookbook app?”
Ask about what they are currently doing:
“Hey, Mom, how do you currently do yoga?” or “where do you find and store recipes currently? Do you have any favorite cookbooks?”
You can also see whether they have an existing problem or even care about the topic.
Try to uncover something that isn’t widely known already
Before even considering asking consumers the questions you want answers to, consider if the answer is already known. For example, what instrument is most popular for people to learn? Is that something we need to do a survey on, or could we google the answer and easily know that the answer is guitars?
“You should never do first-hand research for something you could have googled,” Rob said. “If you can find an easier tool to get to the learning goal, then take the easier tool.”
Read next: Research recruitment: Finding the right research participants for your study
Could you explain why you need their feedback?
People want to feel that they, in particular, can help with a problem, which can be addressed in the way we ask questions. So, for example, we might say:
As a loyal customer of xyz brand, how do you currently…
OR
As somebody who regularly buys xyz products, what’s your process to…
“They want to know you value their time and aren’t a time waster,” Rob said. “Saying ‘can I pick your brain,’ feels like a time waster.”
The result of good research questions
Good questions avoid leading consumers to fluff answers, compliments, and hypotheticals.
Rob said that compliments are nice to hear, but were they meant? And how can they be helpful from here going forward?
“It probably means you showed your ego, and that ego is very detectable,” Rob said. “You are not going to be talking about a product, and a person would say, ‘I would never use this in a million years. It’s the dumbest thing I’ve ever heard of.’ And even if they did, they are still bad at predicting their future behavior. They may be wrong. I would have said that about Instagram, and then I still used it every day for ten years.”
Good questions lead to answers that do not start with “I might use that…” or “I probably would…”
“Those are the trickiest because they feel like data,” Rob said. “It’s so tempting. I’ve seen many startup pitches where they say, ‘we’ve surveyed 1,000 customers, and 95 percent said they would definitely use this product.’ And then, low and behold, the startup launches, and they get zero percent usage. That’s because it’s really easy to say ‘yes’ on a survey, but it’s harder to learn a new product and see where it fits into your life.”
And even if you get some of these less-useful types of answers during the research, remember that certain types of responses and data aren’t that helpful.
“It’s hard to keep a conversation on track,” said Rob. “So some of those hypotheticals and compliments will naturally surface. This trick is just to treat it as noise.”
But even when you ask good questions, you still have to analyze the answers correctly.
“Understanding really what the customer is saying,” said Colin Shaw, of Beyond Philosophy. “Rather than just saying, ‘Oh, we surveyed our customers – did I like this product? They liked Product B. So let’s sell them Product B. I’m afraid life isn’t as simple as it sounds.”
Be specific
When you look at the research results, be specific about what you learned. Don’t just say, “oh, that went well, or they loved it,” said Rob.
“Don’t assign an emotion to it,” he said. “What you want to do is pull out specific things they said. Like they tried it this way or that way, and nothing worked. So if you get those specific data points, you can pull them out, which is the value.”
At the end of the day, we want consumer insights that help us make decisions that help the business. Understanding the goal of our questions and how to ask them in the best way can get us started down the right path of building something consumers want.
Like this article? Read more like it here.
Listen to our market research podcast
