TY - JOUR
T1 - Practicing what we preach (and sometimes study)
T2 - Methodological issues in experimental laboratory research
AU - Nichols, Austin Lee
AU - Edlund, John E.
N1 - Publisher Copyright:
© 2015 American Psychological Association.
PY - 2015/6/1
Y1 - 2015/6/1
N2 - At the core of any causal claim is an experimental study, and laboratory research often provides the most valid arena for conducting experiments. Laboratory experiments allow researchers to provide internal validity to any phenomena they examine. However, researchers must consider and address certain methodological concerns when conducting lab research. Before participants ever agree to participate, forces are beginning to affect how they will respond to experimental manipulations. These and additional concerns continue during and after the completion of the experiment, yet many researchers do not fully understand their implications. As serious as these issues have always been, the introduction of social media and Internet data collection only highlight the importance of understanding and addressing methodological issues in laboratory experiments. In the following article, we discuss potential methodological flaws in laboratory research, examine research relevant to these problems, review evidence that suggests problems actually do contaminate our data, and suggest areas for future research. We conclude by providing recommendations for eliminating or alleviating these problems in future research, considering the role of replication studies as a post-hoc means of detecting these issues, and apply our conclusions to experimental research performed outside or a laboratory. In all, we believe that addressing these concerns in experimental research and practice will strengthen the validity of our conclusions, the replicability of our findings, and science as a whole.
AB - At the core of any causal claim is an experimental study, and laboratory research often provides the most valid arena for conducting experiments. Laboratory experiments allow researchers to provide internal validity to any phenomena they examine. However, researchers must consider and address certain methodological concerns when conducting lab research. Before participants ever agree to participate, forces are beginning to affect how they will respond to experimental manipulations. These and additional concerns continue during and after the completion of the experiment, yet many researchers do not fully understand their implications. As serious as these issues have always been, the introduction of social media and Internet data collection only highlight the importance of understanding and addressing methodological issues in laboratory experiments. In the following article, we discuss potential methodological flaws in laboratory research, examine research relevant to these problems, review evidence that suggests problems actually do contaminate our data, and suggest areas for future research. We conclude by providing recommendations for eliminating or alleviating these problems in future research, considering the role of replication studies as a post-hoc means of detecting these issues, and apply our conclusions to experimental research performed outside or a laboratory. In all, we believe that addressing these concerns in experimental research and practice will strengthen the validity of our conclusions, the replicability of our findings, and science as a whole.
KW - Experiment
KW - Experimenter
KW - Laboratory
KW - Methodology
KW - Research
UR - http://www.scopus.com/inward/record.url?scp=84933177836&partnerID=8YFLogxK
U2 - 10.1037/gpr0000027
DO - 10.1037/gpr0000027
M3 - Article
AN - SCOPUS:84933177836
SN - 1089-2680
VL - 19
SP - 191
EP - 202
JO - Review of General Psychology
JF - Review of General Psychology
IS - 2
ER -