Good Respondents Gone Bad: MarketTools(R) Proves That Unengaged Respondents Compromise Survey Data

SAN FRANCISCO, April 7 /PRNewswire/ -- New research from MarketTools, Inc., the leading provider of software and services for enterprise feedback management (EFM) and market research, reveals the risk unengaged survey respondents pose to data quality, highlighting the impact of survey design in achieving trustworthy research results. The study proves conclusively that in order to ensure the quality of survey data, researchers must not only remove 'bad' respondents, i.e. duplicate, fraudulent and blatantly unengaged respondents, from their samples, but they must also design surveys that keep 'good' respondents engaged.

While experienced researchers have long assumed that survey design, respondent engagement, and data quality are interrelated, the MarketTools research is the first quantitative measurement of these complex relationships. Using measurements from MarketTools TrueSample® SurveyScore®, a survey engagement scoring technology that is part of the MarketTools TrueSample software, MarketTools researchers answered the following questions with a two-part study:

1. How does survey design impact respondent engagement?
2. How does respondent engagement affect data quality?

Simply put, can "good" respondents driven "bad" by poorly designed and complex surveys lead to a reduction in data quality and less reliable survey results?

Impact of Survey Design Conclusions:

• Survey design does impact engagement, but there are no silver bullet rules for survey design

Respondent engagement is driven by a complex interaction among design variables. There is no single rule that applies in all cases, such as "surveys that require more than 20 minutes to complete result in poor respondent engagement." In fact, MarketTools researchers uncovered several examples of long surveys that had a higher-than-normal respondent engagement rating, and short surveys that had a lower-than-normal respondent engagement rating due to other variables.

• Respondent engagement can be predicted using survey design parameters

MarketTools modeled the impact of more than 20 survey design variables that are within the control of survey designers -- such as survey length, percentage of matrix questions, and total word count --on several respondent engagement measures reflecting the respondents' perception of, and behavior during, the survey. The study revealed that the approximate level of respondent engagement can be predicted based on these design variables, giving researchers the chance to improve their surveys with design modifications before deployment.

Impact of Unengaged Respondents Conclusions:

• Higher abandonment rates in unengaging surveys increase the odds of sample bias
• Respondents answer the same question inconsistently as engagement drops
• Degree of randomness in responses increased noticeably as engagement declines
• Less engaging surveys result in unreliable research results

The study also validates MarketTools TrueSample as the industry standard research quality solution – the only solution that approaches the data quality issue from multiple angles to improve results across any survey platform by removing bad respondents and helping to improve survey design. Unlike other solutions, TrueSample uses a holistic approach to data quality by addressing all of the variables with the potential to negatively impact survey results, such as fake and duplicate respondents, unengaged respondents, and good respondents driven bad by un-engaging surveys.

"This new research confirms the need for researchers to not only remove fake and duplicate respondents from their research sample, but to also design surveys that keep good respondents engaged. The quality of their research depends on it," said John Ouren, EVP and General Manager, Panels and TrueSample, MarketTools, Inc. "With TrueSample SurveyScore®, researchers get a quantitative measurement of the effect their survey design has on respondent engagement and on data quality. This measurement helps researchers understand the impact of their survey design choices, allowing them to modify variables as necessary in order to arrive at the best possible data."

To review the complete research, click here to download the white paper.

The Patent-Pending MarketTools TrueSample Technology

MarketTools TrueSample is the research industry's premier online data quality solution that improves the quality of online research by identifying and removing fake, duplicate and unengaged respondents from research samples.

MarketTools TrueSample combines powerful identity validation, de-duplication and engagement modeling into one comprehensive technological solution that ensures that survey respondents are:

Real – TrueSample uses extensive third-party databases to guarantee that all prospective panelists are who they say they are;

Unique – TrueSample ensures that no respondent can take a survey twice by eliminating both overlaps (panelists in multiple panels) and duplicates (panelists who sign up in one panel more than once);

Engaged – TrueSample's award-winning panelist engagement technology removes speeders and straight-liners, in real time, and provides quantified feedback on survey design via TrueSample SurveyScore®.

More than 100 customers are already benefitting from MarketTools TrueSample, including some of the world's largest consumer packaged goods companies.

About MarketTools, Inc.

MarketTools is the leading provider of software and services for Enterprise Feedback Management (EFM) and Market Research.  The company is focused on providing leading organizations the actionable customer insights they need to make better business decisions that lead to high-value business impact.  As the first company to make online surveys widely available on the Web, MarketTools continues its market-leading position by providing the broadest range of powerful, accurate and integrated customer insight technologies that empower companies to become the most customer-centric organizations in their industries. MarketTools' premier portfolio of technology-based insight brands includes CustomerSat™, MarketTools.com™, TrueSample®,  Zoomerang®, ZoomPanel® and ZoomPanel Tech™.

MarketTools is a privately held company with corporate headquarters in San Francisco and European headquarters in London. For more information, please visit www.markettools.com.

All trademarks are the property of their respective owners.

SOURCE MarketTools, Inc.