Concerns About Respondent Experience – Seriously?

Mobile Graphic

Recent surveys of research providers and buyers by Greenbook (GRIT Q3-Q4 2016; GRIT Consumer Participation Report March 2017) raised a serious red flag for me. The most important factors in the design of a research investigation are trust between provider and buyer and the quality of the samples that we use. This makes sense right? Factors that are actually at the heart of MSI International’s core values.

However, what doesn’t make sense is that both providers and buyers of marketing research place low importance on the respondent experience. The three lowest factors were…

  • That participants speak positively of the experience, become evangelists for marketing research,
  • Participants are compensated based on their contribution, and
  • Participants have a positive impression of the industry after completing a survey

Now this finding is nothing new and the GRIT surveys have identified the low concern for the respondent experience previously. But seriously, can we really place importance on trust and data quality if we ignore the people who are providing the data? Sample quality is extremely important and MSI goes to great lengths to minimize the inherent skews in many of the sampling sources available to the industry. But research quality goes well beyond our sample sources.

At MSI’s 10th anniversary (we’re now pushing towards our 30th), we had a company celebration and launched a tagline – “In order to get a better answer you need to ask a better question.” That simple line is still relevant today. It really doesn’t matter how much attention you pay to sample design and sample source quality if your questions are broken.

Today this concept is even more important. Our experience has found over the past 4-5 years the percentage of respondents completing on a device other than a PC has increased dramatically. Recently we have found roughly 26% are completed on a smartphone and 7% are completed on a tablet. Have you tried to complete a survey designed to be executed on a PC on a Smartphone? We have and it’s not easy to do.

At MSI International we’re working to make all of our surveys Mobile Friendly if not Mobile First. All of these efforts will result in a higher respondent experience and better quality data.

But this challenge is not an easy one. In many cases, we need to reduce question wording and modify how the question is presented to make the survey friendly. While that in and of itself is an easy task, it becomes extremely difficult when there are norms or the study is part of an ongoing tracking study.

Within the next month MSI will field a study that executes a questionnaire across 3 formats…traditional, Mobile Friendly and Mobile first. We expect to get rigorous data on the impact of the changes on response rates, dropout rates and the results themselves. With this data in hand we can start having a fact based discussion about the respondent experience and its impact on data quality. More to come in future blogs – stay tuned – seriously!