Cookies

We use cookies to ensure that we give you the best experience on our website. You can change your cookie settings at any time. Otherwise, we'll assume you're OK to continue.

Durham University

Research & business

View Profile

Publication details for Professor Nick Vivyan

Kuhn, Patrick M. & Vivyan, Nick (2021). The Misreporting Trade-off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries. Political Analysis

Author(s) from Durham

Abstract

To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase non-strategic misreporting. We provide the first empirical assessment of this trade-o between strategic and non-strategic misreporting.
We field list experiments on election turnout in two different countries, collecting measures of respondents' true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of non-strategic reporting errors. For both list experiments, partition validation reveals non-strategic misreporting that
is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.