Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Published: Proceedings of the National Academy of Sciences, 2023

Authors: Christoph Huber, Anna Dreber, Jurgen Huber, ..., #ManyDesignsTeam

Abstract: Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently it has been tested in experimental studies yielding an inconclusive body of empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity—systematic variation in effect sizes across various conceivable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. A meta-analysis of the pooled data provides evidence of an adverse effect of competition on moral behavior, but the effect size is small. The novel crowd-sourced design of our study allows for a clean identification and estimation of the systematic variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity—estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs—, indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving towards much larger data collections on various conceivable experimental designs testing the same hypothesis.

Keywords: competition, moral behavior, metascience, generalizability, experimental design