Increasing nonresponse error by reducing nonresponse rates? Investigating the biasing effect of methods and procedures aimed at increasing response rates in Web-based access panel surveys
Final Report Abstract
Evaluating strategies to increase response rates and their effects on sample composition was the main goal of this research project. Two experiments were conducted to test this question. The initial sample was recruited via the German General Social Survey (ALLBUS) in 2008. The focus of the first experiment, implemented as a web and mail survey, was the manipulation of several types of prenotification and feedback. Different incentive strategies were the focus of the second web experiment. Both experiments show nearly identical research findings according to their effect on sample composition. None of the tested strategies to increase response rates (i.e., prenotification, feedback, and incentives) had a significant influence on sample characterisfics. No differences between the experimental groups with respect to the measured personality and value variables, civic duty orientation, and sociodemographic variables were found. Only on the first recruitment level (Internet users vs. non-Internet users) a slight personality bias and differences in sociodemographics occur. However, there was no interaction effect between these sample characteristics and the strategies used on subjects answering behavior. Internet users are younger, better educated, marginally more open minded, less conscientious, and more post-materialistically oriented. On the other hand, the findings support significant strategies to increase response rates like postal prenotification, reminder actions, and prepaid and postpaid monetary incentives. Nonrespondents to the first experiment can effectively be interviewed by using prepaid incentives in the follow up survey without loss of data quality. For already survey-minded persons postpaid or other incentives strategies are more efficient, because they tend to participate already without getting an incentive. There are three main results of this project: First, the findings show that strategies to increase response rates can be used without fearing a biasing effect on sample characteristics, which in turn might influence the answering behavior. Second, this study replicates earlier work showing different sample compositions of Internet users compared to non-Internet users. Finally, we demonstrate the efficiency of postal prenotification, reminders, and monetary incentives.
Publications
-
(2009, May). Strategies for increasing response rates in Web and mail surveys: Effects on sample composition. 64th Annual AAPOR Conference 2009, American Association for Public Opinion Research (AAPOR 09), May 14-17, 2009, Fort Lauderdale, FL, USA
Haas, I., Galesic, M., Couper, M., Bosnjak, M. & Bandilla, W.
-
(2009, Sept.). Effects of monetary incentives on participation in a two-wave online survey. Workshop on Internet Survey Methodology (ISM 09), September 17-19, 2009, Bergamo, Italy
Bandilla, W. & Haas, I.
-
(2009. April). Strategies for increasing response rates and their effects on sample composition. General Online Research Conference 2009 (GOR 09), April 6-8, 2009, Vienna, Austria
Haas, I., Bosnjak, M., Couper, M., Bandilla, W. & Galesic, M.
