John Protzko
University of California, Santa Barbara
Co-Authors: Daniel J. Simons
How do scientists update their beliefs? An investigation of scientists engaged in data collection
Do scientists adjust their beliefs based on the observed evidence, or do they stick with their prior beliefs? Few studies have examined how beliefs change over the course of a study, from developing the protocol to collecting and analyzing data. Scientists who were contributing to one or more of six multi-lab replication projects were asked to estimate their degree of belief in the original effect and to estimate the true effect size at three timepoints: before data collection, after learning their own results, and after learning the results of all of the other laboratories contributing replication studies. We examine how beliefs changed in response to data, whether scientists are disproportionately influenced by their own study results relative to those of other labs, and whether people optimally update their prior beliefs according to the observed evidence.