Developing Standards for Post-Hoc Weighting in Population-Based Survey Experiments

Annie Franco, Neil Malhotra, Gabor Simonovits, L. J. Zigerell

Research output: Contribution to journalReview Articlepeer-review

Abstract (may include machine translation)

Weighting techniques are employed to generalize results from survey experiments to populations of theoretical and substantive interest. Although weighting is often viewed as a second-order methodological issue, these adjustment methods invoke untestable assumptions about the nature of sample selection and potential heterogeneity in the treatment effect. Therefore, although weighting is a useful technique in estimating population quantities, it can introduce bias and also be used as a researcher degree of freedom. We review survey experiments published in three major journals from 2000-2015 and find that there are no standard operating procedures for weighting survey experiments. We argue that all survey experiments should report the sample average treatment effect (SATE). Researchers seeking to generalize to a broader population can weight to estimate the population average treatment effect (PATE), but should discuss the construction and application of weights in a detailed and transparent manner given the possibility that weighting can introduce bias.

Original languageEnglish
Pages (from-to)161-172
Number of pages12
JournalJournal of Experimental Political Science
Volume4
Issue number2
DOIs
StatePublished - 2017
Externally publishedYes

Keywords

  • PATE
  • SATE
  • Survey experiment
  • external validity
  • representativeness
  • transparency
  • weighting

Fingerprint

Dive into the research topics of 'Developing Standards for Post-Hoc Weighting in Population-Based Survey Experiments'. Together they form a unique fingerprint.

Cite this