Loading...
Using experimental vignettes to study how survey methods and findings affect the public's evaluation of public opinion polls: Considering a dual process approach
Holbrook, Allyson L. Lavrakas, Paul J. Johnson, Timothy P. Crosby, Andrew Polskaia, Polina Wang, Xiaoheng Hu, Xiaoyan Kapousouz, Evgenia Cho, Young Ik Silber, Henning
Holbrook, Allyson L.
Lavrakas, Paul J.
Johnson, Timothy P.
Crosby, Andrew
Polskaia, Polina
Wang, Xiaoheng
Hu, Xiaoyan
Kapousouz, Evgenia
Cho, Young Ik
Silber, Henning
Citations
Altmetric:
Other Names
Location
Time Period
Advisors
Original Date
Digitization Date
Issue Date
2025-10-20
Type
Article
Genre
Keywords
Dual processing models,Motivated reasoning,Public opinion,Survey climate,Vignette research
Subjects (LCSH)
Citation
Holbrook, A. L., Lavrakas, P. J., Johnson, T. P., Crosby, A., Polskaia, P., Wang, X., … Silber, H. (2025). Using Experimental Vignettes to Study how Survey Methods and Findings Affect the Public’s Evaluation of Public Opinion Polls: Considering a Dual Process Approach. Survey Research Methods, 19(3), 335–353. https://doi.org/10.18148/srm/2025.v19i3.8261 (Original work published October 15, 2025)
Abstract
Understanding how the public thinks about and understands survey findings is an important part of understanding the role of surveys in policymaking and democracy more broadly. In this paper, we examine the results of three vignette experiments conducted within representative sample surveys in which the methodology and results of a hypothetical public opinion survey assessing support for specific public policy proposals was manipulated. Respondents’ beliefs about the accuracy of the survey described in the vignette experiment, and their beliefs about whether or not it should be considered by policymakers were measured. We used both manipulated (e.g., the methodological rigor of the survey described in the vignette) and measured (e.g., respondents’ opinions on the proposed policy about which public opinion was measured in the vignette) to test four different models that could be used to explain public evaluations of public opinion surveys: (1) the rational actor model, which suggests that people will evaluate more methodologically rigorous surveys more positively; (2) the science literacy model, which suggests that people high in science literacy will evaluate more methodologically rigorous surveys more positively than will people low in science literacy; (3) the motivated reasoning model, which suggests that people will evaluate surveys more positively when the survey results are consistent with their prior opinions than when they are inconsistent; and (4) a dual process model approach, which suggests that people will evaluate more methodologically rigorous surveys more positively only when they are both able and motivated to do so. We found some support for the scientific literacy and motivated reasoning models, but these findings were qualified by an interaction between factors associated with respondent motivation, ability, and survey methodology rigor that strongly supports the dual process model perspective.
Table of Contents
Description
This is an open access article under the CC BY license.
Publisher
European Survey Research Association