As it turns out, scientific studies with phony findings are not as uncommon as they should be. And far too often, bad journalism results in the uncritical reporting of these phony findings.
- 1. Start With a Wrong Assumption
- 2. Throw Out the Data You Don’t Like
- 3. Oops, We Can’t Detect the Chemical in Question
Whenever you hear that there is none of a chemical in something, it’s time
to ask: What’s the detection limit?
- 4. Findings That Aren’t Statistically Significant
- 5. Design the Study to Get the Results You Want
- 6. All of the Above
This has always been true, Ruth, from the beginning of science. There is a saving grace, however, or actually two. The first of these happens once the huckster publishes his study and others learn of it. Somewhere there is likely to be someone else interested in this same field or phenomenon and he or she will attempt to replicate the experiment or study. When the attempt at replication fails, as it is certainly likely to do, and when other attempts do likewise, both the hypothesis and the person asserting its truth are liable to suffer and deservedly so.
The other mechanism is that of peer review, the process of presenting one's proposed theory before experts in that field, in an attempt to verify and legitimize a given proposal. If the process mentioned above falls on its face, then the situation here will likely be no less disastrous for the proposer and the concept in question.
Pseudo-scientific organizations like the Discovery Institute and others have occasionally claimed to have peer-reviewed studies supporting their claims, but far more often than not, those review committees are woefully lacking in participants other than those who already side with a given proposition. Indeed, confirmation bias runs rampant through such organizations. Their credibility remains limited to their own ilk and finds little or none outside of it.
Science has checks upon checks upon checks for this very reason. BS does get out occasionally (witness cold fusion) ... but it doesn't last very long.
As Loren says, sooner or later, science is self-correcting. Even if much later, it's still almost infinitely better than religion.
I've always loved science because I like knowing how things work. Combine that with being an honest person, and one that wants to know the truth, and I'm disappointed to find I've been rather naive most of my life about how many 'scientists' are less than rigorous, and even dishonest.
Interesting point, Karim. *nod*
Peer review can do well or poorly.
In a magazine I was flipping through a couple of weeks ago, a story said that in an area of knowledge where change is happening, an article that explains the change, even objectively, is likely to get negative reviews from older peers and positive reviews from younger peers.
Where little is changing, peer review works better.