Shooting Fish in a Barrel

Someone recently posted a scientific article on our Facebook page which purportedly demonstrates that Roundup can be damaging to earthworms at concentrations that would typically be used in a field situation. Wow. Scary. I mean really, if we’re damaging earthworms when we apply Roundup, then that lends fuel to the emotional fires that rage against this pesticide. But is that really what this article shows?

It’s unfortunate, but most of you will not be able to see the article that I’m writing about because you won’t have access to the journal in which it was published. Here’s the abstract though.

http://link.springer.com/article/10.1007/s11270-014-2207-3

Basically what the authors did was to place worms in small pots, expose the pots to different concentrations of a commercial formulation of Roundup, and measure how the worms fared over time (about a month and a half). Unsurprisingly, the worms not exposed to Roundup performed better than the worms exposed to the Roundup.

After reading the above paragraph you might think that this is an open and shut case. Roundup is bad for worms, potentially leading to “local extinction” of these animals in agricultural fields (that’s the authors’ wording).

It’s not that simple. The authors are stretching well beyond the data, and the research has some issues, most of which could be cleared up by better, more thorough reporting.

First, let’s take a look at some of the problems that this paper has in terms of reporting its materials and methods. You may think this is picky, but it’s not. It’s fundamental to figuring out how valid the reported results are. From the materials and methods as they were written it is impossible to figure out exactly what was done in terms of watering the pots (we know soil moisture was kept at 80%, but we don’t know how. Watering? With what?). We don’t know what the ground plant materials were that were added to the pots (Lima beans?). We know that pots were placed into 1m X 1m X 0.60 m containers, but we don’t know how many pots were placed into each container or whether pots were randomized by treatment within each container. Sure, we could make assumptions – but in a well written scientific paper we shouldn’t have to. Would knowing these things affect how the worms performed in the Roundup treatment versus the no Roundup treatment? In a word, yes. The watering regime in particular might very well alter the results of this study.

That’s enough of that. Now let’s take a look at my BIG PROBLEM with this study. Six worms were placed into small (28cm X 14cm), half-filled pots and treated, or not treated, with Roundup.

Let me offer an extreme analogy to explain why this is such a problem. Let’s say that you want to see whether shooting bullets into the ocean will kill all of the fish that live there. To test the theory you grab a 50 pound fish and you stick it in a 5 gallon bucket. The tail is hanging out, the fins are flapping, water is getting all over the place. Then you shoot the bucket. Dead fish. You do this 50 more times. Each time, dead fish. You conclude that shooting bullets into the ocean is indeed a threat to fish and may lead to local extinction. Right?

Wrong.

From this study you can conclude that bullets can kill fish. That’s an easy conclusion to make. You cannot conclude that shooting bullets into the ocean will kill all the fish there. Now, if we hired a swat team to fire bullets into the ocean and all the fish were killed, well then we could make that conclusion. Would that actually happen though? No way of knowing unless we try it. I suspect the ocean would retain its fish – but I’m just hypothesizing. (Quick FYI – high velocity bullets lose so much of their speed when they hit water that they wouldn’t be lethal to fish after traveling about 3-4 feet).

There are any number of studies out there that FORCE target organisms to be exposed to whatever chemical is being tested (that is basically what is being done here). These studies CAN show that the chemicals tested MAY affect the target organism. They CANNOT show that the target organism IS AFFECTED IN A GIVEN ENVIRONMENT. You need to test the chemical in that environment to figure that out.

To give an example of how you might test the effects of Roundup against worms in an agricultural environment: Take an acre of agricultural field, divide it into six sections. Treat three with Roundup and control weeds in the other three sections with hand weeding. Sample the sections every two or three weeks after Roundup application to see how the worms are doing.

Now, my final problems with this paper. Much of it is related to other, already published studies. This, in and of itself, is no problem. It is good that there are many studies on this topic. The problem is that most of these studies weren’t mentioned in this article. When I read a scientific article I count on its authors to put their study into context for me so that I can see where it belongs in the already existing collection of related literature. Without referencing these older papers the authors do us a disservice. I’m not going to list out all of the studies, but if you go to scholar.google.com and type in earthworm and glyphosate you’ll see what I mean.

I believe that any experiment from which data can be extracted should be published. I think that the authors of this article had every right to publish it. However, as a scientist, I think that there are enough problems with the reporting of this article, particularly the materials and methods, that, as it is currently presented, I can’t extract much of value. I certainly can’t reach the sweeping conclusions that its authors do.

10 thoughts on “Shooting Fish in a Barrel”

  1. Thanks for the summary. Another attempt by the anti-Roundup crowd to provide Dr. Oz and Joe Mercola with more sensationalist pabulum.

  2. I agree with your thoughtful analysis. The “organic” horticulturalists among us are never going to win over our chemically-minded peers with poorly conducted studies. I see so many other possible flaws with the study that I’m thinking it is counterproductive to those of us that do not depend on concentrated chemcials like RoundUp and the mesaage we are wanting to portray. Thanks for your thoughtful critique.

  3. Whether an experiment is about Roundup or anything else – I appreciate the instruction about how good experiments are designed, and how results can be explained. Some people really don’t understand scientific process and we can hope that articles like this one will find their way to people who need instruction. And would probably welcome that instruction.

  4. Great piece, Jeff. I’m getting very tired of well meaning and seemingly rational people’s ability to believe only what they want to believe.

  5. Informative piece, thank you.
    I can say that most people I talk to about these issues rely totally on news articles reporting on a study, not on the study paper itself. Journalism school doesn’t require the student have the skills to understand scientific methodology. Any sane person would be crazy to go to their barber for hair transplants, but many people are comfortable believing a journalist without assessing their credibility first. Go figure!

  6. Here was nice piece in Nature from July 2014. It did mention different varieties of Earthworms were more effected than others. It also dealt with effects on mycorrhizae, but not directly, but more indirectly through the plants which caused reduction and reduced spore viability. In any event it was interesting. The earthworms slowed down their activity. They grew fatter in deeper soil where plant roots and hyphae died and provided abundant food, but made them sluggish and less active as a result of not moving throughout the system. Anyway it was interesting.

    http://www.nature.com/srep/2014/140709/srep05634/full/srep05634.html

  7. I would be willing to accept larger containers, say 10 L each or sonething. This is routine with earthworm studies and has been compared to field plots. Small containers are not acceptable, except to make conclusions about small containers. But treatment volume and concentration are as important as is the a.i. and adjuvant (formulation) rate per unit soil, especially when the effect may be due to surfactants, etc. This study needed to be backed up with linked studies with larger containers, and small plots, confined underground. And what about reproduction? It is a big factor, even in 6 weeks. Soil a.i. concentration mean and variance are crucial, as are moisture content, pH trend, and a basic microbial assessment of some kind for the experimental units. Simple measures are possible.

Leave a Reply to commonweeder Cancel reply