This is not good:
A former researcher at Amgen Inc has found that many basic studies on cancer — a high proportion of them from university labs — are unreliable, with grim consequences for producing new medicines in the future.
During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 “landmark” publications — papers in top journals, from reputable labs — for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development.
Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature.
Posted in science
Jonah Lehrer has some more thoughts on the Decline Effect:
The first letter, like many of the e-mails, tweets, and comments I’ve received directly, argues that the decline effect is ultimately a minor worry, since “in the long run, science prevails over human bias.”
Lehrer then quotes Feynman who discusses the famous 1909 oil-drop experiment and explains why it took so long for scientists to zero on the correct measure for the charge of the electron:
Why didn’t they discover that the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan’s value they didn’t look so hard.
As Lehrer notes, this is yet another example of the “selective reporting in science.” But Feynmann was trying to make another point:
he warned the Caltech undergrads to be rigorous scientists, because their lack of rigor would be quickly exposed by the scientific process. “Other experimenters will repeat your experiment and find out whether you were wrong or right,” Feynman said. “Nature’s phenomena will agree or they’ll disagree with your theory.”
But Lehrer is quick to puncture the obvious naivety associated with this claim:
Remember my concern about naïve realism?
Naïve realism is the conviction that one sees the world as it is and that when people don’t see it in a similar way, it is they that do not see the world for what it is. Ross characterized naïve realism as “a dangerous but unavoidable conviction about perception and reality”. The danger of naïve realism is that while humans are good in recognizing that other people and their opinions have been shaped and influenced by their life experiences and particular dogmas, we are far less adept at recognizing the influence our own experiences and dogmas have on ourselves and opinions. We fail to recognize the bias in ourselves that we are so good in picking out in others.
Of course, many people might be tempted to dismiss this as being rather insignificant, given that science has provided a means to “see the world for what it is.” Not so fast. I encourage you to read Jonah Lehrer’s article, The Truth Wears Off : Is there something wrong with the scientific method?
Lehrer explains the Decline Effect, where scientific findings are reported and with time, it becomes harder and harder for others to replicate the findings. The problem is widespread and there appear to be many factors that bring about this phenomenon. For those who have heard me talk about confirmation bias in the past, you might enjoy this example:
Some people don’t like McDonalds. To support their agenda, they often claim that science shows there is something unnatural about the burgers and fries. Consider this video:
Of course, there is nothing scientific about the video. All you have are a set of observations across time. And yes, while the food does not decay, there is no reason to think these observations mandate the belief that there is something wrong or unnatural about the food. What’s more, it is not going to help to come up with some mathematical analysis that argues it is extremely unlikely for the food to resist decay, therefore something unnatural is happening.
If you want to make a scientific case against McDonalds, then you need to go beyond the realm of observation and speculation. You need to come up with specific hypotheses and then test them with experiments. That is, you would need to consider a range of possible explanations for the food not decaying and then test each explanation. And the experiment is key, because you need to design the experiment to include both positive and negative controls. This is the only way to eliminate confirmation and disconfirmation bias.
Posted in science
Most people love to use the words “science” and “scientific” when advocating their own views. And wouldn’t ya know it? Almost every time they use those words, oddly enough, their views just happen to align with “science.” Clearly, people recognize science as an authority in our culture and seek to posture as if that authority sides with them.
But when you ask people to define what they mean when they use that word, you’ll find that the word comes with all sorts of meanings.
Over at Telic Thoughts, Bilbo continues to insist that ID is science by stressing “the fact that in the not so very distant past, ID was considered science in the field of biology.”
There are three prisms by which we might perceive science – philosophy, history, and sociology. From the philosophical prism, people try to come up with some ideal and abstract definition of science, but this effort has failed, as no one has generated a definition that has become universally embraced. As such, there are lots of personal definitions of science and I see them all the time all over the place. I myself stand in the tradition of Jacques Monod, who wrote:
Hence it is through reference to our own activity, conscious and projective, intentional and purposive-it is as makers of artifacts-that we judge of a given object’s “naturalness” or “artificialness.”
It is my view that, in the final analysis, all design inferences must appeal to this intuition and that an ultimate reliance on a subjective judgment call is not science. Since we have been conditioned by a culture of scientism, many people find this to be a very bitter pill to swallow (What? Science is not the road to Truth?!). Me? I simply recognize this as another limitation of science – if life was designed, the design would exist within the blind spot of science.