There are two ways that scientists judge the impact of an article: the quality of the journal; and the level of interest shown by the research community in the findings. Cell, Nature and Science have high impact factors and are among the most prestigious journals to publish groundbreaking research, while the number of times an article is cited by others is an indirect measure of the importance of the paper.
The Center for Open Science is a not-for-profit organization that is affiliated with the University of Virginia in Charlottesville, VA and is currently conducting “The Reproducibility Project: Cancer Biology”. They have compiled a list of 50 high-impact papers from 2010 to 2012 for replication, where the topics covered range from the biology of metastasis to drug resistance. The corresponding authors have been contacted and informed that the project and its partners are planning to conduct independent experiments to see if the article’s results can be replicated, and they might require assistance from the authors on the materials and experimental design.
Not surprisingly, some authors are skeptical about the benefits of the project. Richard Young of MIT, who published a paper in Cell in 2012 on a protein called c-Myc that stimulates tumour growth, says, “I am a huge fan of reproducibility. But this mechanism is not the way to test it.” He will be unsurprised if the project’s results match his own – a European lab has already published a replication of his study – but if it does not, he believes there may be a lack of expertise in the replicating lab.
Timothy Errington is managing the reproducibility project and he says his team has worked hard to ensure they duplicate the original studies. They are publishing the research protocols in the open-access journal eLife and plan to publish the findings for all 50 papers by 2017. Errington is adamant the project will benefit the field of cancer biology, and comments, “Some see this as a threat, a way to disprove something. That’s not what this is about.”
Motivation for the project comes from firms like Amgen and Beyer whose scientists have been unable to reproduce the findings in greater than 75% of published papers. Elizabeth Iorns a cancer biologist who launched the Reproducibility Initiative argues that the research community should be less concerned about damaged reputations and threats to research funding and focus instead on assisting pharmaceutical companies in their search for new cancer treatments. (accompanying illustrations are by Davide Bonazzi, © Science).