A bad dream for those crafting online survey questionnaires might go as follows…
Panel members take an online survey critiquing a previous online survey in which they participated, and one that you wrote. The results are overwhelmingly negative. The panel members hated the experience and admit they offered slapdash information that, as an effect, will actually hinder your research more than help.
You don’t have to wake up, as it hasn’t happened—yet. However, if you want to avoid this reality and gain high-quality survey responses, which we highly recommend, it’s wise to understand two key concepts that will help craft the best possible questionnaire.
Respondent fatigue
Also known as survey fatigue, respondent fatigue refers to the mental state when respondents become weary during an online survey, to the point their answers become rushed and even dishonest—thereby reducing statistical accuracy. Furthermore, an article in Great Brook explains:
“Fatigue means that those with extreme views are more likely to respond, leading to a serious survey bias, known as non-response bias. That is, those who don’t respond likely have different views from those who do. The survey data that winds up in the survey database don’t properly reflect all customers’ views. The data are biased.”
Respondent fatigue can be considered more of a direct threat than just a negative impact from online surveys. The New York Times reported that respondent fatigue has caused “declining response rates over the last decade.”
A chief reason for respondent fatigue is the length and wordiness of a survey, and science supports this. A study found survey fatigue often sets in after 20 minutes into a survey. It states: “They found survey respondents exert less effort and spend less time thinking about their answers as respondents get deeper into the survey.” However, our own experience shows that survey fatigue starts to set in after 15 minutes. Of course, this also depends on the complexity of the survey, which is why it’s critical to keep survey questions as short and to the point as possible, regardless of the survey topic.
LEARN ABOUT: Panel Respondents
Respondent fatigue not only affects data quality, but other areas of data collection that are just as critical. For example, an 8-10 minute survey will often yield a survey drop out rate of about 8% or less. After 15 minutes, drop out rates can hover around 15% or more. Although regarded as a positive survey feature, this is often the time that respondents start to look for the survey progress bar if the survey is not moving quickly enough. In essence, this feature can be more of a curse than a blessing when it comes to survey fatigue.
Length, clarity, and approachability are effective tools to combat respondent fatigue, but ultimately they are not enough. Our second notion, empathy map, is also essential to create a positive survey answering experience.
Empathy
Putting yourself in a respondent’s shoes goes a long way when trying to get into their hearts and minds. Sharing their interests is not enough, but understanding that online respondents are real people with busy lives and little spare time (just like you). The respondents are just as essential as the researchers, the managers, the higher-level decision-makers who are seeking data or looking to use the data in some way, and all are equally just as part of the process.
The person(s) writing the survey instrument should ask themselves if they could sit through that survey for 25-30 minutes. Research practitioners should design survey instruments that are fun and engaging, regardless of the topic. They should test their surveys over and over again to identify the fatigue points in the survey. This is usually the area where data integrity is compromised.
There are sensible solutions that QuestionPro Audience offers to cultivate empathy with respondents:
- Be communicative from the beginning—as in explain the number of questions, privacy policies, and purpose of the online survey. Communicating survey results during the survey is also an effective way of holding respondent interest.
- Ask yourself if every question is absolutely necessary—if you were to personally take it in a real-time event (for example, on your smartphone or while waiting for the train)—and get rid of anything else.
- Is it worth their time? Research practitioners often complain about data quality, but rarely advocate to provide adequate incentives to research participants. Ask yourself if your research budget allocates enough space to provide proper incentives.
In addition, there are practical techniques that can increase empathy and improve the overall architecture of a questionnaire. For example, minimizing matrices questions is a vital tactic to producing efficient questionnaires, as they tend to exhaust respondents and produce sloppy answers. Some practitioners even refer to extensive matrices as “the laziest type of survey writing.”
Of course, there are also other issues that compromise online surveys—such as response acquiescence (the tendency to agree with survey questions regardless of content) and respondent bias (the inability to answer survey questions due to perceived social pressure). These issues can be moderated by various means that online survey providers should be able to recommend; they are commonly addressed by tweaking questions and providing some safeguards in the programming of the questionnaire software platform.
However, having empathy is an alpha and omega for the best possible questionnaire, certainly for decreasing respondent fatigue and ultimately yielding high quality survey responses. This should not be surprising, in the end, as empathy is the solution to so many other problems, in both the virtual and real world.
