Ryan Brutger, Joshua D. Kertzer, Jonathan Renshon, Chagai M. Weiss Abstraction in Experimental Design

Annotation

Political scientists designing experiments often face the question of how abstract or detailed their experimental stimuli should be. Typically, this question is framed in terms of tradeoffs relating to experimental control and generalizability: the more context introduced into studies, the less control, and the more difficulty generalizing the results. Yet, we have reason to question this tradeoff, and there is relatively little systematic evidence to rely on when calibrating the degree of abstraction in studies. We make two contributions. First, we provide a theoretical framework which identifies and considers the consequences of three dimensions of abstraction in experimental design: situational hypotheticality, actor identity, and contextual detail. Second, we field a range of survey experiments, varying these levels of abstraction. We find that situational hypotheticality does not substantively change experimental results, but increased contextual detail dampens treatment effects and the salience of actor identities moderates results in specific situations.