Kleber Neves
Instituto de Bioquímica Médica Leopoldo de Meis, Universidade Federal do Rio de Janeiro / Brazilian Reproducibility Initiative
Co-Authors: Pedro Tan, Olavo Amaral
How much reproducibility do we want? An app and model for understanding replication
How much reproducibility do we want in a given field? The goal is probably in the middle: too much and we are stifling risk-taking, too little and we are wasting resources. To address this issue, we developed a computational model and an associated R Shiny App where one can explore reproducibility in different scenarios – varying sample sizes, bias and publication incentives. In the model, hypotheses are tested in experiments, generating a literature, upon which replication initiatives take place. First, as a calibration of the model, we ran simulations to successfully replicate the analytical results from the scenarios in Ioannidis’s “Why most published research findings are false”. We also evaluated how the rate of true positives in the literature relates to different measures of reproducibility. In the future, we plan to release the the app to the public as an interactive online tool, as well as to expand the model to work with continuous effect sizes instead of dichotomous ones.