What is the cost of the reproducibility crisis and the waste in research? This question came up in a recent Tweet pointing out the possible falsification of images in a key study on Alzheimer’s disease in 2006 that led to considerable subsequent research. As Alex Holcombe pointed out “Even without fraud, many avenues that biologists pursuing for years or decades are likely totally spurious due to poor reproducibility standards.” In reply someone asked about the costs of poor reproducibility.
Calculating the cost of poor reproducibility might be complex, but we had worked out the closely related costs of avoidable waste in research.
Avoidable waste occurs at four successive stages: the choice of research questions; the quality of research design and methods; the adequacy of publication practices; and the quality of reports of research. The estimated losses are roughly 50% loss at stages 2, 3, and 4 implying that the dividends from tens of billions of dollars of investment in research are lost each year because of correctable problems. A series in the Lancet documented how and why this avoidable waste occurs, but the calculation 85% research being wasted is best set out in our blog.
The world currently invests over US$100 billion every year in supporting biomedical research, and this results in an estimated 1 million research publications annually. This investment in research is laudable. But regardless of who sponsors research, something needs to be done to protect this investment to improve health from the avoidable waste in producing and reporting research. Some of the problems are readily correctable for a minimal investment.
Fortunately there are several groups actively working to reduce this avoidable research waste. As three examples:
1/ The EVIR funders forum founded in 2017 is a forum of over 40 research funders to help health-related research funders increase the value of their research.
2/ The EQUATOR Network for Enhancing the QUAlity and Transparency Of health Research – develops and disseminates the reporting guidelines (CONSORT, PRISMA, etc) to improve research reporting and transparency.
3/ The Evidence Based Research network aims to reduce waste in research by promoting: (a) No new studies without prior systematic review of existing evidence, (b) Efficient production, updating and dissemination of systematic reviews.
More effort is needed, and work will be slow but the pace of that work seems to be increasing.
Healthy Evidence 