Concerns About Respondent Experience – Seriously?

Mobile Graphic

Recent surveys of research providers and buyers by Greenbook (GRIT Q3-Q4 2016; GRIT Consumer Participation Report March 2017) raised a serious red flag for me. The most important factors in the design of a research investigation are trust between provider and buyer and the quality of the samples that we use. This makes sense right? Factors that are actually at the heart of MSI International’s core values.

However, what doesn’t make sense is that both providers and buyers of marketing research place low importance on the respondent experience. The three lowest factors were…

  • That participants speak positively of the experience, become evangelists for marketing research,
  • Participants are compensated based on their contribution, and
  • Participants have a positive impression of the industry after completing a survey

Now this finding is nothing new and the GRIT surveys have identified the low concern for the respondent experience previously. But seriously, can we really place importance on trust and data quality if we ignore the people who are providing the data? Sample quality is extremely important and MSI goes to great lengths to minimize the inherent skews in many of the sampling sources available to the industry. But research quality goes well beyond our sample sources.

At MSI’s 10th anniversary (we’re now pushing towards our 30th), we had a company celebration and launched a tagline – “In order to get a better answer you need to ask a better question.” That simple line is still relevant today. It really doesn’t matter how much attention you pay to sample design and sample source quality if your questions are broken.

Today this concept is even more important. Our experience has found over the past 4-5 years the percentage of respondents completing on a device other than a PC has increased dramatically. Recently we have found roughly 26% are completed on a smartphone and 7% are completed on a tablet. Have you tried to complete a survey designed to be executed on a PC on a Smartphone? We have and it’s not easy to do.

At MSI International we’re working to make all of our surveys Mobile Friendly if not Mobile First. All of these efforts will result in a higher respondent experience and better quality data.

But this challenge is not an easy one. In many cases, we need to reduce question wording and modify how the question is presented to make the survey friendly. While that in and of itself is an easy task, it becomes extremely difficult when there are norms or the study is part of an ongoing tracking study.

Within the next month MSI will field a study that executes a questionnaire across 3 formats…traditional, Mobile Friendly and Mobile first. We expect to get rigorous data on the impact of the changes on response rates, dropout rates and the results themselves. With this data in hand we can start having a fact based discussion about the respondent experience and its impact on data quality. More to come in future blogs – stay tuned – seriously!

Faces of Consumption: Smartphones The Incidental Finding

CELL_usageHave you ever intended to look up something in particular, only to end up discovering something else? To draw a parallel to marketing research, this would be similar to initiating a study with a set of objectives only to find an unexpected outcome. In research, we refer to the discovery of unexpected outcomes as incidental findings. In spite of incidental findings being unintentional, they often lead to rich insight.

Recently, I began drilling down on mountains of interesting data regarding our Faces of Consumption initiative when I stumbled upon incidental findings that were intriguing.

Every Picture Tells a Story

Social_Media_ReportIn the words of Rod Stewart – every picture tells a story. At MSI we’re constantly bantering about social media, which sites will make it, which ones will die a slow death. So we decided to take a deeper look at a variety of social and technology trends in 2013. And we’re going to tell that story with pictures. Our first foray was a look at the social media sites with the largest followings. And in order to tell the story, we used a tried but true method of having consumers associate “faces” of consumers with the sites. A total of 10 pictures were used to get out the faces and personality of the major social media sites.

And what we found may surprise you…

Personalizing Your Marketing Research

Customized_researchAs users of marketing research intelligence, we recognize the merits of custom marketing research clearly outweigh a one-size-fits-all approach. Studies designed around precise objectives yield outcomes specifically addressable to those objectives. With all that said, a recent experience as a survey participant prompted me to think of opportunities to take our customization even further. This began thanks to a restaurant that I visit frequently sending me a questionnaire where I became a highly involved participant. Little did I know that the information I was about to provide would be leveraged into action for my INDIVIDUAL benefit. Following is how the process played out as well as ideas going forward on how you can personalize marketing research to your customers’ benefit.

The Ever Evolving Social Network – Perceptions of Today


In spite of social networks appearing to be new on the scene, they have evolved in dramatic ways. Thinking back to the early days of social networks, they were comprised predominantly of male engineers. Subsequently, a new social network came on the scene, first referred to as The Facebook which was comprised of college students. Fast forward to today and people from all walks of life are on social networks. In fact, they have become part of our fabric. Some may even go as far as expressing that certain social networks have become less cool given that grandmas and grandpas embrace them. Nevertheless, it is through social networking websites that we communicate with others by:

Pretesting…a lost art?


There was a time when collecting quantitative data, more often than not, involved a conversation. And the conversation was a game where the respondent seemed to hold all of the cards. A lengthy interview, a poorly worded question, a missing response option – all represented excuses for the respondent to dismiss you by hanging up or otherwise ending the interview.

Can you tell I was an interviewer?

At that time, phone pretests often involved a painful process of listening to hours of rejections and hang-ups…but the result was invariably worth the price – a survey that more accurately collected the intended data.

The Tortoise Wins Again. Faster Data Collection Is Not Always Better.

At the risk of sounding like my grandfather, I’ve seen a lot of change in my career in terms of the speed at which we can deliver insights. Back in the day, we used to send tabs and reports to clients via snail mail. No, we didn’t use the Pony Express, but compared to today’s technology standards, it seems as though we weren’t far from that. ☺

I know I’m stating the obvious when I say that the advent of online interviewing brought a dramatic change in the speed at which we can collect data and deliver insights. And with lightning-fast capabilities sometimes comes the perception that we should collect vast amounts of data as fast as we can, maybe in just a couple of days at most. I understand that perception. And I understand the need for speed to insight in today’s world. But is faster always better?

And the winner is…results from our recent Oscar poll.

MSI_Academy_Awards_winnerAnother Hollywood award season culminated Sunday night with the biggest show of the year, the Academy Awards. From the red carpet parade of designer gowns to the over-the-top show production, from the “genuine” heart-felt (if not incoherent) acceptance speeches of the winners to the gracious applause from the losers – and not to mention Jennifer Lawrence’s embarrassing trip up the stairs – the Academy Awards brought its audience a wide range of emotions that only Hollywood can.

For those of us in the market research industry, the Oscars also gave us another chance to see how well the so-called experts could predict the winners. At MSI, we were curious to see how well our audience fared against the Nate Silvers and David Rothschilds of the world. We asked our social media followers a few questions on who they wanted to win, who they thought should win, and who they predicted would win in the Best Actor, Best Actress, and Best Picture categories.

With over 100 people responding, MSI followers on average watched 2 out of the 9 movies nominated for Best Picture. Over one-quarter (29%) did not watch a single nominated movie (if The Avengers was nominated that number would be much closer to 100%), but why let that stop people from making predictions or sharing their opinions? Does it matter that so many people did not watch these most of these movies? Would we see a bias towards Silver Linings Playbook, set locally in Philadelphia and starring hometown actor Bradley Cooper?

Let’s find out. Without further ado, here are the results (queue the music as we open the envelope)…

The Oscars Survey

February 24th is the biggest Hollywood event of the year. Although we’d like to think we’re above it all, let’s admit it, we’re all a little hopeful of an embarrassing trip up the stairs to collect the award or some incoherent babble we can laugh at in an acceptance speech. Really, who doesn’t love Jack Nicholson smirking in the front row or question why we all need to listen to the tight security guarding the results by PricewaterhouseCoopers? Celebrity, glitterati, whatever you call it, it’s completely superficial but yet we can’t help ourselves. It’s fun! It’s an event! We love movies!

So, unlike our usual efforts where we typically solicit opinions among people that are well informed on the topic, here we’re just interested in what you think. No movie going required. Just opinions.

The survey is available at and will be available through 7pm, Sunday, February 24th.

Chalk another one up for Beer Bloggles shameless pandering!

Check back after the awards when we compare our selections with the “experts”!

Hooray for Hollywood!



Valentine’s Day – An obligation

We were curious how our friends, family and clients planned to spend their Valentine’s Day. So we did a quick informal poll with a little bit of humor. Remember this is not science and while we completed 150 interviews we won’t even discuss the inevitable margin of error.

So here are some of the highlights of our quick little poll…

“I do it for her,” probably not surprising, men are much more likely to celebrate Valentine’s Day solely to keep their partner happy. This may explain why women are more likely to claim “Oh is this Valentine’s day week.”


Older folks were more likely to state they actually look forward to this day. Assuming age is an indicator of longer relationships – this suggests that familiarity does not breed contempt.


Many of our followers are traditionalists – they celebrated by flowers, candy and other types of gifts followed by a dinner out yesterday. There was some time shifting to weekend celebrations.

Romance beyond the living room didn’t appear to be on the agenda yesterday. Men clearly see an opportunity to share a good bottle of wine with their partners, but there was little indication of amorous activities from men or women. At least no one admitted getting tipsy and seeing what panned out. Further, only 1% participant indicated a romantic getaway was in order.

V_day_getawayWe should note that reading 50 Shades of Grey either alone or with your partner was ignored by all. We’re not sure but this may be an indication of the literary sophistication of our audiences.