Shooting Fish in a Barrel

Someone recently posted a scientific article on our Facebook page which purportedly demonstrates that Roundup can be damaging to earthworms at concentrations that would typically be used in a field situation. Wow. Scary. I mean really, if we’re damaging earthworms when we apply Roundup, then that lends fuel to the emotional fires that rage against this pesticide. But is that really what this article shows?

It’s unfortunate, but most of you will not be able to see the article that I’m writing about because you won’t have access to the journal in which it was published. Here’s the abstract though.

http://link.springer.com/article/10.1007/s11270-014-2207-3

Basically what the authors did was to place worms in small pots, expose the pots to different concentrations of a commercial formulation of Roundup, and measure how the worms fared over time (about a month and a half). Unsurprisingly, the worms not exposed to Roundup performed better than the worms exposed to the Roundup.

After reading the above paragraph you might think that this is an open and shut case. Roundup is bad for worms, potentially leading to “local extinction” of these animals in agricultural fields (that’s the authors’ wording).

It’s not that simple. The authors are stretching well beyond the data, and the research has some issues, most of which could be cleared up by better, more thorough reporting.

First, let’s take a look at some of the problems that this paper has in terms of reporting its materials and methods. You may think this is picky, but it’s not. It’s fundamental to figuring out how valid the reported results are. From the materials and methods as they were written it is impossible to figure out exactly what was done in terms of watering the pots (we know soil moisture was kept at 80%, but we don’t know how. Watering? With what?). We don’t know what the ground plant materials were that were added to the pots (Lima beans?). We know that pots were placed into 1m X 1m X 0.60 m containers, but we don’t know how many pots were placed into each container or whether pots were randomized by treatment within each container. Sure, we could make assumptions – but in a well written scientific paper we shouldn’t have to. Would knowing these things affect how the worms performed in the Roundup treatment versus the no Roundup treatment? In a word, yes. The watering regime in particular might very well alter the results of this study.

That’s enough of that. Now let’s take a look at my BIG PROBLEM with this study. Six worms were placed into small (28cm X 14cm), half-filled pots and treated, or not treated, with Roundup.

Let me offer an extreme analogy to explain why this is such a problem. Let’s say that you want to see whether shooting bullets into the ocean will kill all of the fish that live there. To test the theory you grab a 50 pound fish and you stick it in a 5 gallon bucket. The tail is hanging out, the fins are flapping, water is getting all over the place. Then you shoot the bucket. Dead fish. You do this 50 more times. Each time, dead fish. You conclude that shooting bullets into the ocean is indeed a threat to fish and may lead to local extinction. Right?

Wrong.

From this study you can conclude that bullets can kill fish. That’s an easy conclusion to make. You cannot conclude that shooting bullets into the ocean will kill all the fish there. Now, if we hired a swat team to fire bullets into the ocean and all the fish were killed, well then we could make that conclusion. Would that actually happen though? No way of knowing unless we try it. I suspect the ocean would retain its fish – but I’m just hypothesizing. (Quick FYI – high velocity bullets lose so much of their speed when they hit water that they wouldn’t be lethal to fish after traveling about 3-4 feet).

There are any number of studies out there that FORCE target organisms to be exposed to whatever chemical is being tested (that is basically what is being done here). These studies CAN show that the chemicals tested MAY affect the target organism. They CANNOT show that the target organism IS AFFECTED IN A GIVEN ENVIRONMENT. You need to test the chemical in that environment to figure that out.

To give an example of how you might test the effects of Roundup against worms in an agricultural environment: Take an acre of agricultural field, divide it into six sections. Treat three with Roundup and control weeds in the other three sections with hand weeding. Sample the sections every two or three weeks after Roundup application to see how the worms are doing.

Now, my final problems with this paper. Much of it is related to other, already published studies. This, in and of itself, is no problem. It is good that there are many studies on this topic. The problem is that most of these studies weren’t mentioned in this article. When I read a scientific article I count on its authors to put their study into context for me so that I can see where it belongs in the already existing collection of related literature. Without referencing these older papers the authors do us a disservice. I’m not going to list out all of the studies, but if you go to scholar.google.com and type in earthworm and glyphosate you’ll see what I mean.

I believe that any experiment from which data can be extracted should be published. I think that the authors of this article had every right to publish it. However, as a scientist, I think that there are enough problems with the reporting of this article, particularly the materials and methods, that, as it is currently presented, I can’t extract much of value. I certainly can’t reach the sweeping conclusions that its authors do.