The first letter, like many of the e-mails, tweets, and comments I’ve received directly, argues that the decline effect is ultimately a minor worry, since “in the long run, science prevails over human bias.”
Lehrer then quotes Feynman who discusses the famous 1909 oil-drop experiment and explains why it took so long for scientists to zero on the correct measure for the charge of the electron:
Why didn’t they discover that the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan’s value they didn’t look so hard.
As Lehrer notes, this is yet another example of the “selective reporting in science.” But Feynmann was trying to make another point:
he warned the Caltech undergrads to be rigorous scientists, because their lack of rigor would be quickly exposed by the scientific process. “Other experimenters will repeat your experiment and find out whether you were wrong or right,” Feynman said. “Nature’s phenomena will agree or they’ll disagree with your theory.”
But Lehrer is quick to puncture the obvious naivety associated with this claim:
But that’s not always the case. For one thing, a third of scientific papers never get cited, let alone repeated, which means that many errors are never exposed. But even those theories that do get replicated are shadowed by uncertainty. After all, one of the more disturbing aspects of the decline effect is that many results we now believe to be false have been replicated numerous times.
He then adds:
However, I think the decline effect is an important reminder that we shouldn’t simply reassure ourselves with platitudes about the rigors of replication or the inevitable corrections of peer review. Although we often pretend that experiments settle the truth for us—that we are mere passive observers, dutifully recording the facts—the reality of science is a lot messier. It is an intensely human process, shaped by all of our usual talents, tendencies, and flaws.
The primary lesson I personally take away from all this is to at least be more skeptical of scientific discoveries that are rooted in one lab or a small number of labs. This doesn’t mean I should deny those discoveries, but means only that I acknowledge them in an extremely tentative manner.
Now, I don’t think anyone would get much mileage out of using the decline effect as a reason to doubt the evidence for evolution. That phenomenon has been studied too long, too widely, and in too much detail to think it will one day fade with further study. In fact, one could make the case that the evidence for evolution has become stronger, the exact opposite of the decline effect.
On the other hand, the decline effect seems to map nicely to abiogenesis research. Here is a field of research that is produced by a small number of researchers highly motivated to find positive results that, when reported, are not widely replicated or replicated at all. In fact, when it comes to the various theories about abiogenesis, they come across more as fads than zeroing in on the truth. For example, anyone remember Sidney Fox and his supposedly ground-breaking research on proto-cells? He created quite the buzz about this back in the 1980s. But today, his research is largely ignored. Why is that? Could it be that others tried to replicate his findings and failed? Who knows? Negative results are rarely reported. All we know is that his research has somehow fallen off the radar screen.
So, when it comes to abiogenesis research I think it both prudent and fair to request evidence that the decline effect does not or will not apply.