Improving your survey screening questions

Surveys are a critical part of your project work. A great survey can distill and quantify opinions which, in turn, create truly actionable insights. A great survey requires a keen understanding of how a survey is built, how to reach your ideal audience and how to ask them the right questions that don't lead the respondent where you are hoping they go.

In these posts, we outline how you can empower your surveys, improve the quality of research you get and ultimately generate quality data that adds true value. proSapient are always on hand to support you with our unique platform and incredible survey team.

The Screening questions

When you create a survey, the respondents are specific. You may be looking to survey coffee drinkers or key purchasers of underwater radar equipment used to detect schools of fish. Regardless of how general or how niche, it is important to make sure the respondents you receive are those you want to hear from, rather than those who want to earn a few dollars.

1. Survey panels cannot instantly 'target' your niche requests

Regardless of what you have heard, survey panel companies generally cannot target your niche respondent set. Some consumer panels may have incremental data such as where the user has shopped or which streaming service they use but B2B panels have limited information on their panels aside from the general industries they work in, the geographies they live in and how much they earn. You can find this data in Panel Books which are usually published by survey panel companies. Find examples here, here and here.

Instead, they can point your survey to a population that contains your ideal respondents, but the difference between your great survey and a botched survey is the screening questions.

2. The screening questions

In order to create a great survey, you must create screening questions. The survey will be sent to irrelevant people. It will be sent to people who enjoy earning money from clicking through surveys. It will also be sent to the right people. How can you design screening questions to reach only those whose insights add value?

Use open questions

Do you work in the salmon farming industry?

If you are given a link to a survey that offers $50 for 10 mins work and this is the first question, are you going to click yes or no? It's immediately obvious that the survey is looking for salmon farmers. For $50 and 10 minutes' work, of course you are clicking yes! Pocket that dollar and tell the world why salmon is the cow of the 21st century.

Don't be afraid to be completely off-topic

Which industry do you work in? Stationary, off-shore gas, aquaculture, consumer goods, or marketing

A good set of screening questions should give the respondent no clue as to the topic of conversation. The above question makes it equally likely that each topic could be chosen and makes it less likely that a fake respondent can enter the survey

Rule of three

If the respondent gets the first question right, challenge them again. If you have provided 5 options there is still a 20% chance of a fake respondent getting into your survey. Make sure you continue with open questions!

What type of aquaculture are you presently working in? Crab farming, salmon farming, fish food, salmon farming

This second question reduces the likelihood of a false respondent to 5%. A third will bring this under 1%.

Incidence

When you receive pricing for a particular survey, the pricing is based on the incidence rate. In other words, if I poll my survey panel at random, what % of these respondents are going to be relevant to the survey. The smaller the incidence rate, the more expensive the survey.

The lower the incidence rate, the more stringent your screening questions need to be. If only 1/100 panelists are relevant to your survey but there is a 25% chance of entering the survey by answering randomly, your survey won't add value.

Let us help you

proSapient's dedicated survey team is here to help you create a survey that adds value. We won't let you spend money on a survey that won't bring value and we will act as a partner to help you discover unique insights to power your research.

Our supercharged survey platform makes the whole process seamless and brings a collaborative online experience, real-time results and a host of other great features.

Send us an email and try us out! -> Hayley@prosapient.com