The Impact of Result Significance on the Assessment of Replication Accuracy: An Experimental Study

Abstract

The study aimed to examine the impact of manipulating the order of presentation of studies with opposing conclusions on the assessment of the accuracy of research. In this registered parallel experimental study, participants from the general population were randomly given descriptions of studies in which replications of psychological research produced results that showed either an effect or no effect for interventions, and were asked to assess the accuracy of the conclusions of the described studies. The study was conducted in a virtual environment. Analyses included 194 individuals, 97 in each experimental group. The intervention in this study consisted of different research conclusions randomly assigned to participants by the testing program. The results suggest that
a higher percentage of participants who were shown a replication with a significant effect assessed the replication as more accurate compared to the initial research (n = 71, 73.20%), than participants who were shown a replication
that did not show a statistically significant effect (n = 53, 54.64%). The results suggest a potential bias in the order of presentation when interpreting replication studies because
the accuracy assessment depended on the findings of the research. Future qualitative research should investigate factors associated with the interpretation of replications.