Reproducibility and robustness of economics and political science research
Research output: Contribution to journal › Research article › Contributed › peer-review
Contributors
- Tilburg University
- University of Oxford
- University of Guelph
- University of Melbourne
- Universite Paris Dauphine
- University of York
- Deakin University
- Autonomous University of Barcelona
- European University Institute, San Domenico di Fiesole
- University of Bologna
- RWI - Rhenish-Westphalian Institute for Economic Research Essen
- Stockholm School of Economics (SSE)
- American University of Armenia
- Georgetown University in Qatar
- Durham University
- Dongbei University of Finance and Economics
- Brock University
- Université du Québec à Montréal
- University of California at Berkeley
- Cornell University
- Texas Tech University
- University of Calgary
- Oregon State University
- George Washington University (GWU)
- University of Oslo
- University of Queensland
- Lancaster University
- Federal Urdu University of Arts, Science and Technology
- University of Warwick
- University of California at Los Angeles
- Luxembourg Institute of Socio-Economic Research
- University of Innsbruck
- Université Laval
- University of Strathclyde
- University of California at San Diego
- University of Bonn
- Risk Software Technologies
- Charles University Prague
- Czech Academy of Sciences
- Monash University
- University of Southampton
- Carleton University
- University of Montreal
- Columbia University
- University of Toronto
Abstract
Science aspires to be cumulative. Reproducibility efforts strengthen science by testing the reliability of published findings, promoting self-correction, and informing policy-making1. Computational reproductions, whereby independent researchers reproduce the results of published studies, are an essential diagnostic tool2-10. Such efforts should have greater visibility11-16. However, little social science reproduction and robustness has been conducted at scale10,13,17-23. Here we reproduced original analyses and conducted robustness checks of 110 articles that were published in leading economics and political science journals with mandatory data and code sharing policies17,18. We found that more than 85% of published claims were computationally reproducible. In robustness checks, our reanalyses showed that 72% of statistically significant estimates remain significant and in the same direction, and the median reproduced effect size is nearly the same as the originally published effect size (that is, 99% of the published effect size). Additionally, 6 independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness did not correlate with author characteristics or data availability.
Details
| Original language | English |
|---|---|
| Pages (from-to) | 151-156 |
| Number of pages | 6 |
| Journal | Nature |
| Volume | 652 |
| Issue number | 8108 |
| Publication status | Published - 2 Apr 2026 |
| Peer-reviewed | Yes |
External IDs
| PubMed | 41922705 |
|---|---|
| ORCID | /0000-0003-4378-0847/work/212492652 |