Failure to replicate

This is not good:

A former researcher at Amgen Inc has found that many basic studies on cancer — a high proportion of them from university labs — are unreliable, with grim consequences for producing new medicines in the future.

During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 “landmark” publications — papers in top journals, from reputable labs — for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development.

Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature.

Google Mystery

What does it mean?

More Thoughts on the Decline Effect

Jonah Lehrer has some more thoughts on the Decline Effect:

The first letter, like many of the e-mails, tweets, and comments I’ve received directly, argues that the decline effect is ultimately a minor worry, since “in the long run, science prevails over human bias.”

Lehrer then quotes Feynman who discusses the famous 1909 oil-drop experiment and explains why it took so long for scientists to zero on the correct measure for the charge of the electron:

Why didn’t they discover that the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan’s value they didn’t look so hard.

As Lehrer notes, this is yet another example of the “selective reporting in science.”  But Feynmann was trying to make another point:

he warned the Caltech undergrads to be rigorous scientists, because their lack of rigor would be quickly exposed by the scientific process. “Other experimenters will repeat your experiment and find out whether you were wrong or right,” Feynman said. “Nature’s phenomena will agree or they’ll disagree with your theory.”
But Lehrer is quick to puncture the obvious naivety associated with this claim:

The Decline Effect

Remember my concern about naïve realism?

Naïve realism is the conviction that one sees the world as it is and that when people don’t see it in a similar way, it is they that do not see the world for what it is. Ross characterized naïve realism as “a dangerous but unavoidable conviction about perception and reality”. The danger of naïve realism is that while humans are good in recognizing that other people and their opinions have been shaped and influenced by their life experiences and particular dogmas, we are far less adept at recognizing the influence our own experiences and dogmas have on ourselves and opinions. We fail to recognize the bias in ourselves that we are so good in picking out in others.

Of course, many people might be tempted to dismiss this as being rather insignificant, given that science has provided a means to “see the world for what it is.”  Not so fast.  I encourage you to read Jonah Lehrer’s article, The Truth Wears Off : Is there something wrong with the scientific method?

Lehrer explains the Decline Effect, where scientific findings are reported and with time, it becomes harder and harder for others to replicate the findings.  The problem is widespread and there appear to be many factors that bring about this phenomenon. For those who have heard me talk about confirmation bias in the past, you might enjoy this example:

Food science

Some people don’t like McDonalds. To support their agenda, they often claim that science shows there is something unnatural about the burgers and fries. Consider this video:

Of course, there is nothing scientific about the video. All you have are a set of observations across time. And yes, while the food does not decay, there is no reason to think these observations mandate the belief that there is something wrong or unnatural about the food. What’s more, it is not going to help to come up with some mathematical analysis that argues it is extremely unlikely for the food to resist decay, therefore something unnatural is happening.

If you want to make a scientific case against McDonalds, then you need to go beyond the realm of observation and speculation. You need to come up with specific hypotheses and then test them with experiments. That is, you would need to consider a range of possible explanations for the food not decaying and then test each explanation. And the experiment is key, because you need to design the experiment to include both positive and negative controls. This is the only way to eliminate confirmation and disconfirmation bias.

Using the ‘science’ word

Most people love to use the words “science” and “scientific” when advocating their own views. And wouldn’t ya know it? Almost every time they use those words, oddly enough, their views just happen to align with “science.” Clearly, people recognize science as an authority in our culture and seek to posture as if that authority sides with them.

But when you ask people to define what they mean when they use that word, you’ll find that the word comes with all sorts of meanings.

The Three Prisms for Science

Over at Telic Thoughts, Bilbo continues to insist that ID is science by stressing “the fact that in the not so very distant past, ID was considered science in the field of biology.”

There are three prisms by which we might perceive science – philosophy, history, and sociology.  From the philosophical prism, people try to come up with some ideal and abstract definition of science, but this effort has failed, as no one has generated a definition that has become universally embraced.  As such, there are lots of personal definitions of science and I see them all the time all over the place.  I myself stand in the tradition of Jacques Monod, who wrote:

Hence it is through reference to our own activity, conscious and projective, intentional and purposive-it is as makers of artifacts-that we judge of a given object’s “naturalness” or “artificialness.”

It is my view that, in the final analysis, all design inferences must appeal to this intuition and that an ultimate reliance on a subjective judgment call is not science.  Since we have been conditioned by a culture of scientism, many people find this to be a very bitter pill to swallow (What?  Science is not the road to Truth?!).  Me?  I simply recognize this as another limitation of science – if life was designed, the design would exist within the blind spot of science.

Thar She Blows Again!

A few weeks back, I showed you how PZ Myers misleads people with the word ‘science’ by falsely making it synonymous with a critical, rational examination.  Well, Jerry Coyne recently did the same thing.

Coyne begins by recognizing there are limitations to science:

Science, Scientism, ID, and the Matrix

[I’ve combined all the recent science/design entries together to make it easier to read.  However, I did not have the time to thoroughly edit, so some parts might seem a little repetitive and awkward.]

A portion of Douglas J. Futuyma’s textbook Evolution is available on the web – the chapter that describes natural selection and adaptation.  The NCSE describes Futuyma as the “Distinguished Professor in the Department of Ecology and Evolution at the State University of New York at Stony Brook.”  In other words, he is a highly respected, mainstream evolutionary biologist.  His textbook (the linked chapter is from the second edition) is a widely used, mainstream text on evolution which was reviewed by dozens and dozens of other scientists.  While it may seem trivial to point this out, we will soon see it is a very important consideration.

I want you to consider a key excerpt from the text, entitled Design and mechanism:

The complexity and evident function of organisms’ adaptations cannot conceivably arise from the random action of physical forces. For hundreds of years, it seemed that adaptive design could be explained only by an intelligent designer; in fact, this “argument from design” was considered one of the strongest proofs of the existence of God. For example, the Reverend William Paley wrote in Natural Theology (1802) that, just as the intricacy of a watch implies an intelligent, purposeful watchmaker, so every aspect of living nature, such as the human eye, displays “every indication of contrivance, every manifestation of design, which exists in the watch,” and must, likewise, have had a Designer.

Supernatural processes cannot be the subject of science, so when Darwin offered a purely natural, materialistic alternative to the argument from design, he not only shook the foundations of theology and philosophy, but brought every aspect of the study of life into the realm of science. His alternative to intelligent design was design by the completely mindless process of natural selection, according to which organisms possessing variations that enhance survival or reproduction replace those less suitably endowed, which therefore survive or reproduce in lesser degree. This process cannot have a goal, any more than erosion has the goal of forming canyons, for the future cannot cause material events in the present. Thus the concepts of goals or purposes have no place in biology (or in any other of the natural sciences), except in studies of human behavior. – (p. 282; emphasis not added).

Let’s pull out the key point to make it crystal clear:

My Inner Felix

In the comments section of a previous entry, chunkdz posed an excellent question to me.  So let me use this opportunity to stand on my soapbox.

First, you are asserting that your approach is not science but simply “open-ended curiosity, guided by critical thinking and intellectual honesty”

But you simultaneously acknowledge Haack’s assertion that science is not limited to the scientific method but that it should include “everyday modes of inquiry”:

“…scientific inquiry is contiguous with everyday empirical inquiry. Everyday knowledge is supplemented by evolving aids that emerge throughout the process of honest inquiry. These include the cognitive tools of analogy and metaphor that help to frame the object of inquiry into familiar terms.”

Sounds like a description of your own Design Matrix. An “everyday mode of inquiry”, as the article says, which relies upon analogy, metaphor, and honest inquiry.

If Susan Haack says your approach is contiguous with scientific inquiry and therefore part of science – then why should you so vehemently disagree?

Yes, Haack’s description is very close to what The Design Matrix is all about and I am delighted that chunkdz sees this.  But I don’t think she is saying science is not limited to the scientific method but that it should include “everyday modes of inquiry.”  If she is, I am not.  What I am saying, and I think she is saying, is that the scientific method is not limited to the domain of science as the scientific method is part of everyday modes of inquiry.  That is, since we can all employ the scientific method in everyday modes of inquiry, we should not fall victim to the message of those who preach scientism  – “Either your views are part of science and of great value or they are little more than subjective fantasy and of little value.”

So why is it that I do not consider my approach as part of science?