Failure to replicate

This is not good:

A former researcher at Amgen Inc has found that many basic studies on cancer — a high proportion of them from university labs — are unreliable, with grim consequences for producing new medicines in the future.

During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 “landmark” publications — papers in top journals, from reputable labs — for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development.

Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature.

Here’s some more:

Other scientists worry that something less innocuous explains the lack of reproducibility.

Part way through his project to reproduce promising studies, Begley met for breakfast at a cancer conference with the lead scientist of one of the problematic studies.

“We went through the paper line by line, figure by figure,” said Begley. “I explained that we re-did their experiment 50 times and never got their result. He said they’d done it six times and got this result once, but put it in the paper because it made the best story. It’s very disillusioning.”


“The surest ticket to getting a grant or job is getting published in a high-profile journal,” said Fang. “This is an unhealthy belief that can lead a scientist to engage in sensationalism and sometimes even dishonest behavior.”

The academic reward system discourages efforts to ensure a finding was not a fluke. Nor is there an incentive to verify someone else’s discovery. As recently as the late 1990s, most potential cancer-drug targets were backed by 100 to 200 publications. Now each may have fewer than half a dozen.

“If you can write it up and get it published you’re not even thinking of reproducibility,” said Ken Kaitin, director of the Tufts Center for the Study of Drug Development. “You make an observation and move on. There is no incentive to find out it was wrong.”


7 responses to “Failure to replicate

  1. Not good at all. 😦

  2. Hey Mike, I wonder if you could do me a favor and find out if I’ve been banned from TelicThoughts. I’ve tried posting comments there, today, with no success. It would be nice to know why.

  3. Obviously this new team did something wrong 47 times and got it right 6 times.

    Case closed- nothing to see here- please move along- we will inform you of those 47 mistakes at our convenience.

    Until then please keep enjoying the landmarks….

    (I am going for my PhD in “science protection”- offered by the NCSE)

  4. Bilbo,

    I asked and no, you have not been banned from posting comments. I do know that sometimes on this blog, your comments get caught in the spam filter. Did you try just making a new account over there?

  5. A new password. So far, my comments from yesterday and today are not showing up.

  6. Lucky for you, they never took back my keys. I went and looked under the hood, and sure enough, they were caught in the spam filter. I let one out, but not all the “test” postings. I would think you should be able to comment now.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s