You’ve probably heard about a project that attempted the replicate 98 psychological studies and found that only a third of them were reproducible, the other two thirds produced different results – sometimes very different, more often only somewhat so.
Though psychology was the subject of this study, you shouldn’t assume the results were unique to that particular field. The are plenty of reports of similar failings and the so-called ‘Decline effect’ in other scientific disciplines.
So why is that? There are a lot of reasons. Research can be poorly designed, based on flawed assumptions, and sometimes an unlucky flukes can create false positives. It is also the sad fact that science is done by humans, and humans are complex things with a lot of motives besides the pure quest of knowledge.
I think the general public often fears that scientists are swayed by money from corporations and/or special interest groups, but my experience in academia is quite different. I’ve never heard anyone concerned they might loose a corporate grant. I have heard lots of people, more-or-less continuously, worrying that if their experiment doesn’t work out they won’t be able to get their PhD, land a job, or get tenure. There is enormous pressure to find something significant, to find an effect, and it matters not at all the political ramifications of that effect. So if you are worried about Monsanto buying off scientists to say GMO are safe to eat, don’t be. Convincing data that GMOs are somehow unsafe to eat would be of enormous significance, completely rewriting what we know about genetics, and would come with huge professional rewards. In my opinion, you should be more concerned that some new study showing that X, Y or Z makes plants grow bigger or yield more is actually the result of fervent, wishful thinking on the part of a grad student desperate for publishable data.
So what’s the solution? There has been a lot of talk in the academic community about making it possible to publish negative results and provide funding to regularly attempt to replicate previous studies. I hope these changes go into effect, as they could make an enormous improvement in the reliability of new findings.
In the mean time, you, as a concerned gardener, should take information supported by only a single, isolated study with a big grain of salt, particularly if it seems to contradict findings from other research. If you go to scholar.google.com and start searching around, make sure you read as much of the research on the topic as you can, so you can differentiate between the intriguing new research that may well be proved wrong and reliable findings that have been sustained by several independent researchers. And always remember that while the scientific process is far from perfect, it is still the best we’ve got.
