Maximize your ROI with these survey design best practices

Survey design best practices in research matter. To get good insights, we have to ask the right questions at the right time. And data quality is impacted by how we kick things off in our surveys. That’s why survey design – the right way – is so important. Every time I launch a survey on the Voxpopme video surveys platform, I see this. The better the questions – qual or quant – the better the content or insights I can get from the respondents.

“There’s just so many little intricate details to build questions that work, that get answers that work, that aren’t offensive, that are easy to understand,” said Annie Pettit, chief research officer at E2E Research on an episode of “Reel Talk: The Customer Insights Show.” “The more you learn about the questionnaire design process, the more you learn it’s not that easy.”

To get us started, it’s good to remember these survey design best practices:

• Document and understand the goals of the survey.
• Know who you are trying to reach.
• Determine what type of survey would be best.
• Understand what the best timing would be. When should we ask respondents?
• Ask good questions.

Keeping these areas top of mind can make it easier to implement them.

“I’m guilty of this, too,” said Jenn Mancusi, CRO at Voxpopme and host of “Reel Talk.” “I just whip up a quick questionnaire, but clearly, it needs some thought for it to be done properly.”

Let’s dive into following good survey design practices in this article.

Read next: 7 steps to conduct research successfully

What’s the definition of a survey?

“A survey gathers knowledge from a large group of people,” Annie said. “You can do a survey via focus group, biometrics, or even social media research.”

Surveys can come in quant and qual versions or include both quant and qual. They can be text-based or video-based, for example. No matter the type of survey you choose, follow this strategic process.

Understand the end goal

Nick Graham, head of global insights and analytics at Mondelez, said on an episode of “Reel Talk” that we need to be aware of internal goals and voices just like we need to bring external voices to the table. That balance is essential to understand before jumping in.

Read more insights from Nick: Do this to be innovative in business

“It requires a lot of thinking and planning, but it saves so much headache for you and time and energy for participants,” Annie said.

Asking some questions ahead of time of some consumers can also help.

Doing some qualitative interviews – through video or remote, for example, can also help researchers write better quantitative questions, said Matthew Handegaard, data scientist at Voxpopme, on “Reel Talk.”

Understand what can be done

Every brand has guardrails of what is possible and what won’t be pursued.

If you know you won’t do specific things with your products, why ask about those things? If you will never use pink packaging, why even bring it up?

“You are just wasting their time and effort,” Annie said.

The questions you ask to make sure they can lead to insights that you or your stakeholders have the authority to act on. Don’t ask about packaging if you aren’t working with the team responsible for the packaging, for example.

Remember, survey participants are people

Jenn reminds us that we must remember that real people are taking the survey. They are not just a sample.

“And it’s not just that we call them ‘sample,'” Annie said. “We talk about the liars and cheaters, which hurts my soul. But the vast majority of respondents are not lab rats, they are not a sample, and they are not cheaters. So we have to flip that state of mind and get rid of that completely.”

Adds Jenn: “Have empathy for the people who help us build our businesses.”

Read next: Why customer empathy should be relatively easy for companies

Also, remember their preferences. For example, Gen Zers likely won’t talk to you on the phone, but they might record a selfie-style video with feedback.

“Gen Z is less willing to put up with bad user experiences, said Jamin Brazil, managing director at Voxpopme. “It’s imperative to ask them questions in the way that makes sense for them.”

Also, keep in mind that not everyone has to take the exact same questions, said Steven Snell, principal survey methodologist at Goldman Sachs.

“It’s about a random sample,” he said. “And it’s not just about randomizing the sample, but also the content. There’s a set of questions that everyone gets – the core questions. And then there’s a random module.”

Read next: Master the Art of Proper Data Quality in Research

Survey design best practices: Starting with good questions

It starts with data quality, Annie said. And to get data quality, the use of red herring questions can be helpful. So what’s a red herring question? These are silly/odd questions that try to see if respondents read and understand the questions.

These come especially in handy in text-based surveys. However, in video surveys, I do, from time to time, see responses where it’s clear the respondent barely read or didn’t understand the question.

Annie says those questions in her survey design help her identify cheaters and people who are having a bad day.

“So when we are including red herrings, we are looking for the people who can’t pay attention right now,” she said. “Maybe they can tomorrow or the next day, but they aren’t having a good day today.”

Read next: Writing amazing video questions for brand and product feedback

When it comes to qual questions, Matthew explained, questions should be:

• broad
• singular
• door openers to more questions and additional insights

Elena Lyrintzis, marketing and culture insights lead, devices and services at Google, said on “Reel Talk” that we shouldn’t ask consumers how they think the world will look in five years. Instead, focus your survey design around their current feelings.

“It’s much more about talking to consumers to see the ‘why’ behind the data,” Elena said. “So, ‘how do you feel about climate change today?’ or ‘how do you feel about your needs being met today?'”

But, also don’t ask questions that you already know the answers to – from other data sources, for example.

“If you can get the data elsewhere, save their time and energy,” Elena said. “They don’t have to repeat it.”

Think outside the box with your questions

Ray Poynter, a 40-year veteran of the industry on “Reel Talk,” recommends that researchers consider asking questions differently. However, we don’t expect different results or better respondent understanding if the way we ask questions doesn’t change.

Survey design best practices: How long should good surveys be?

I know, I know, I also want to get all the answers when I have the chance to talk to consumers. But adding “just one more question” to surveys can quickly turn into adding ten and turn respondents away. Who has the time?

“It needs to be participants first, researchers second,” Annie said. “And that rarely ever happens. It’s all ‘what I need,’ ‘what my client needs.’ So us first and then, oh, by the way, there’s a participant to think about.”

Because surveys have gotten out of hand, response rates decline, and “why we are in the position we are in today,” Annie said.

I love quick surveys. Here’s the brand’s question, and here’s my answer. Done. For example, when I call American Airlines, after the call, the prompt asks something like this: “Push one if you’d recommend this agent, two if you don’t. Feel free to leave a short voicemail with more details.”

But, of course, the right length depends on what you are trying to accomplish and what consumers are willing to spend time on.

“I can’t give a number on where surveys should be,” said Annie. “I would always like them to be under 15 minutes.”

Also, keep in mind the topic at hand. Some topics and questions will keep respondents more engaged than others.

“You could ask me 25 questions about the carpet in the office, and I wouldn’t know how to stay excitedly engaged for the whole time,” Annie said. “It’s not going to happen. So spread the data quality questions and know when people are starting to lose attention.”

Better questions in your survey design

Annie says shorter is better than longer. That includes shorter sentences and even shorter words. Good questions are also written in a way that allows for short answers.

For example, when surveys ask: “Tell us about your experience during your stay at our resort.”

Well, if I’ve stayed there for a week, that’s a lot of stuff to cover. But if the survey asks what worked well or didn’t work well when you rented a canoe, I can answer that relatively quickly.

Read next: 67 Open Questions for Uncovering New Insight with Video Surveys

Understand comprehension levels

Keep in mind that the average American reads at a seventh to eighth-grade level. So if you want your questions to be understood, it’s good to create them roughly at that level.

“And yet when you look at the questionnaires that we write, it’s widespread that they grade at a university level,” Annie said. “That’s because we try to fit every little thing – the caveats, the examples, the instructions. And we end up with one question that’s two to three lines long.”

An easy way to shorten things is to delete or move unnecessary instructions. For example, if there are five radio buttons and you can only select one anyway, why do we need to say in the question: Please select just one answer?

“We don’t need to waste words on that,” Annie said.

Meghan Lacy, director of enterprise expansion at Suzy, said on “Reel Talk” that we should read the questions we just wrote like we are the consumer.

“As a consumer, can I answer this?” she said. “Or, are we thinking about this in too specific of a way?

Common barriers to good survey design

“The biggest barrier is to fit everything in,” said Annie. “It’s simply not the way to go. Instead, we need to make ourselves focus on specific goals, specific outcomes, specific questions.”

To do that, be crystal clear about who your target audience is and then use the most appropriate words for them, Annie said.

“Any survey that’s an hour long, I don’t think you thought that through,” she added. “Did you think you will take action on every single question you put in there? I really doubt it, which means you wasted everybody’s time.”

Moving forward

Matt said that when respondents finish a survey that should lead to even more ideas for future follow-up.

“Any good research is a call for more research,” he said. “Every good research paper ends with ‘here’s the questions I didn’t get answered and that really furthers learning – asking more questions even when you’ve done your work.”

Enjoyed this article? Read more like it here.

Listen to our market research podcast