27.06.2017, 16:55
Cancer studies pass reproducibility test
OREANDA-NEWS Though researchers have had general success reproducing cancer results, studies involving mice have proven difficult to replicate.
A high-profile project aiming to test reproducibility in cancer biology has released a second batch of results, and this time the news is good: Most of the experiments from two key cancer papers could be repeated.
The latest replication studies, which appear today in eLife, come on top of five published in January that delivered a mixed message about whether high-impact cancer research can be reproduced. Taken together, however, results from the completed studies are “encouraging,” says Sean Morrison of the University of Texas Southwestern Medical Center in Dallas, an eLife editor. Overall, he adds, independent labs have now “reproduced substantial aspects” of the original experiments in four of five replication efforts that have produced clear results.
In the two new replication efforts, however, one key mouse experiment could not be repeated, suggesting ongoing problems with the reproducibility of animal studies, says one leader of the Reproducibility Project: Cancer Biology.
The unusual initiative was inspired by reports from two drugs companies that up to 89% of preclinical biomedical studies didn’t hold up in their labs. The project is having contract labs repeat key experiments from about 30 high-impact cancer papers published between 2010 and 2012. Whereas some researchers laud the effort, others have worried that contract labs lack the expertise to perform certain experiments as well as cutting-edge academic research labs and that any failures will unfairly tarnish the field.
In January, critics’ fears were realized when the first five replications came out. Only two studies could be reproduced; one result was negative, and two studies were ruled inconclusive because of problems with mouse tumor models. The findings have led some experts to conclude that biomedicine suffers from a replication crisis.
Now, scientists’ track records seem to be improving. In one of the new studies, a contract lab confirmed a 2010 report in Cancer Cell that mutations in genes called IDH1 and IDH2, found in some leukemias and brain cancers, cause cells to produce a metabolite that spurs cancer growth. The replicators also verified that levels of the metabolite in leukemia cells indicate whether a cancer patient has the IDH mutations. (The original paper’s lead author, Craig Thompson of Memorial Sloan Kettering Cancer Center in New York City, who co-founded a company that is testing IDH drugs in clinical trials, was traveling and unable to comment.)
The second replication study looked at a 2011 Nature paper reporting that a compound called a BET inhibitor, which controls whether genes are activated—can stop a type of leukemia. As in the original study, the compound, I-BET151, killed human leukemia cells in a dish and reduced their numbers in mice that had been injected with the cells. However, unlike in the original paper, these mice did not survive any longer than untreated mice with leukemia.
Several scientists say that result doesn’t invalidate the overall conclusions that I-BET151 works against leukemia. The replication team did the mouse experiment differently, using a lower dose of I-BET151, for example. Given such differences, “I think we should be careful not to make too much of the absence of statistically significant differences in survival as an endpoint,” says Harvard University molecular biologist Karen Adelman, an eLife editor who oversaw reviews of the replication paper.
And cancer biologist Tony Kouzarides of the University of Cambridge in the United Kingdom, who led the original Nature study, says this one negative result “highlights the pitfalls of biological research, namely thatdifferent labs may vary conditions that affect the outcome of a given experiment.”
But Tim Errington of the Center for Open Science in Charlottesville, Virginia, which is co-sponsoring the reproducibility project, counters that the fact that the mouse survival experiment worked only under certain conditions raises questions about whether the paper’s overall findings are “robust.” He adds, “You want this to be generalizable.”
The cancer biology project hopes to finish experiments for another 22 replications by the end of this year, when the grant funding the effort runs out, Errington says.
A high-profile project aiming to test reproducibility in cancer biology has released a second batch of results, and this time the news is good: Most of the experiments from two key cancer papers could be repeated.
The latest replication studies, which appear today in eLife, come on top of five published in January that delivered a mixed message about whether high-impact cancer research can be reproduced. Taken together, however, results from the completed studies are “encouraging,” says Sean Morrison of the University of Texas Southwestern Medical Center in Dallas, an eLife editor. Overall, he adds, independent labs have now “reproduced substantial aspects” of the original experiments in four of five replication efforts that have produced clear results.
In the two new replication efforts, however, one key mouse experiment could not be repeated, suggesting ongoing problems with the reproducibility of animal studies, says one leader of the Reproducibility Project: Cancer Biology.
The unusual initiative was inspired by reports from two drugs companies that up to 89% of preclinical biomedical studies didn’t hold up in their labs. The project is having contract labs repeat key experiments from about 30 high-impact cancer papers published between 2010 and 2012. Whereas some researchers laud the effort, others have worried that contract labs lack the expertise to perform certain experiments as well as cutting-edge academic research labs and that any failures will unfairly tarnish the field.
In January, critics’ fears were realized when the first five replications came out. Only two studies could be reproduced; one result was negative, and two studies were ruled inconclusive because of problems with mouse tumor models. The findings have led some experts to conclude that biomedicine suffers from a replication crisis.
Now, scientists’ track records seem to be improving. In one of the new studies, a contract lab confirmed a 2010 report in Cancer Cell that mutations in genes called IDH1 and IDH2, found in some leukemias and brain cancers, cause cells to produce a metabolite that spurs cancer growth. The replicators also verified that levels of the metabolite in leukemia cells indicate whether a cancer patient has the IDH mutations. (The original paper’s lead author, Craig Thompson of Memorial Sloan Kettering Cancer Center in New York City, who co-founded a company that is testing IDH drugs in clinical trials, was traveling and unable to comment.)
The second replication study looked at a 2011 Nature paper reporting that a compound called a BET inhibitor, which controls whether genes are activated—can stop a type of leukemia. As in the original study, the compound, I-BET151, killed human leukemia cells in a dish and reduced their numbers in mice that had been injected with the cells. However, unlike in the original paper, these mice did not survive any longer than untreated mice with leukemia.
Several scientists say that result doesn’t invalidate the overall conclusions that I-BET151 works against leukemia. The replication team did the mouse experiment differently, using a lower dose of I-BET151, for example. Given such differences, “I think we should be careful not to make too much of the absence of statistically significant differences in survival as an endpoint,” says Harvard University molecular biologist Karen Adelman, an eLife editor who oversaw reviews of the replication paper.
And cancer biologist Tony Kouzarides of the University of Cambridge in the United Kingdom, who led the original Nature study, says this one negative result “highlights the pitfalls of biological research, namely thatdifferent labs may vary conditions that affect the outcome of a given experiment.”
But Tim Errington of the Center for Open Science in Charlottesville, Virginia, which is co-sponsoring the reproducibility project, counters that the fact that the mouse survival experiment worked only under certain conditions raises questions about whether the paper’s overall findings are “robust.” He adds, “You want this to be generalizable.”
The cancer biology project hopes to finish experiments for another 22 replications by the end of this year, when the grant funding the effort runs out, Errington says.
Комментарии