New Study on Research Reproducibility Proposes Fixes for $28 Billion Problem

WASHINGTON, June 9, 2015 /PRNewswire-USNewswire/ -- A new study published in Open Access journal PLOS Biology by the Global Biological Standards Institute (GBSI) and two leading economists estimates that, in the U.S. alone, approximately $28 billion per year is spent on preclinical research that is not reproducible. Analysis of past life science research studies indicates that the cumulative prevalence of irreproducible preclinical research exceeds 50 percent. Because such low reproducibility rates can contribute to both delays and costs of therapeutic drug development, The Economics of Reproducibility in Preclinical Research by Drs. Leonard P. Freedman, Iain M. Cockburn and Timothy S. Simcoe, also outlines a framework for solutions and a plan for long-term improvements in life science research reproducibility rates.

"Our primary goal here is not to pinpoint the exact reproducibility rate but rather to identify root causes of the problem and develop a framework to address the highest priorities," says Freedman who is president of GBSI. "Based on examples from within the life sciences, application of economic theory, and review of lessons learned from other industries, we conclude that community-developed best practices and standards must play a central role in improving reproducibility going forward."

ANALYSIS AND RECOMMENDATIONS
"The problem of reproducibility has been widely discussed," says Cockburn. "To develop our estimate of the current reproducibility rate for preclinical research, we reviewed publicly available data from government sources, industry and analyst reports and scientific articles."

The study included an in-depth analysis of the key components of irreproducibility, and used secondary research to benchmark the potential impacts of improved standards. The authors determined that the causes of irreproducible research could be grouped into four categories that align with the typical stages of study development: (1) biological reagents and reference materials, (2) study design, (3) data analysis and reporting and (4) laboratory protocols. Of these, errors in biological reagents and materials (36% of total) and study design (28% of total) are the two largest contributors to irreproducible spend.

"Improving reproducibility levels will require a measured investment in time and resources," says Cockburn. "We recommend investing in practical solutions and taking immediate steps in the areas where there will be the greatest return on investment."

"While false positives are an inevitable part of scientific research, our study shows that the current level of irreproducibility in preclinical research is very costly," says Simcoe. "By categorizing the root causes of irreproducibility and estimating their relative importance, we can prioritize potential solutions, and ultimately, increase the overall return on public and private investments in research and development."

CALL TO ACTION
Numerous recent articles and papers highlight the challenges of reproducibility in life science research. The Economics of Reproducibility in Preclinical Research calls for a paradigm shift from acceptance of an avoidable problem to the adoption of compelling best practices that support better research and better outcomes.

By encouraging an investment in standards and best practices, the study promotes better scientific research that quickly translates to therapeutics and cures.  This aligns with 21st Century Cures, the U.S. Congressional initiative underway to update laws to support finding new and improved medicines and diagnostics, and keeping America competitive in healthcare and biotech in the 21st century.

"Success requires communicating to build awareness, sharing best practices and identifying practical steps with funding behind them to support scientists in their life-saving mission. It requires that researchers own it, funders fund it, media publicize it and consumers care about it," says Freedman.

Additional resources are available on the GBSI website 
http://www.gbsi.org/publications/the-economics-of-reproducibility/

ABOUT GLOBAL BIOLOGICAL STANDARDS INSTITUTE
GBSI, a non-profit organization, is dedicated to enhancing the quality of biomedical research by advocating best practices and standards to accelerate the translation of research breakthroughs into life-saving therapies.

Media Contact:
Carol Miller
[email protected]
202-306-0130                                                                    

Lynn Blenkhorn
[email protected]
508-851-0930

Citation: Freedman LP, Cockburn IM, Simcoe TS (2015) The Economics of Reproducibility in Preclinical Research. PLoS Biol 13(6): e1002165. doi:10.1371/journal.pbio.1002165

Funding: The authors received no specific funding for this work

Competing Interests: Dr. Simcoe currently works as a Senior Economist for the Council of Economic Advisers (CEA). The CEA disclaims responsibility for any of the views expressed herein and these views do not necessarily represent the views of the CEA or the United States.

Please mention PLOS Biology as the source for this article and include the links below in your coverage to take readers to the online articles.

All works published in PLOS Biology are open access. Use this URL in your coverage to provide readers access to the paper upon publication: http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.1002165

Bios and Headshots:
Leonard Freedman http://www.gbsi.org/about/bio/leonard-p-freedman-phd/ 

Timothy Simcoe
Please use Timothy's Boston University title http://people.bu.edu/tsimcoe/

Iain Cockburn, Bio http://people.bu.edu/cockburn/CockburnBIO.pdf 
Headshot http://images.alfresco.advanstar.com/alfresco_images/pharma/2014/08/21/be3a259e-8f51-410d-b469-46bb72c565b7/Iain.jpg

SOURCE Global Biological Standards Institute