If you’re a regular on the winter conference circuit, you’ve likely seen your fair share of ag research presentations. Most of the regular presenters are adept at translating their research so that the rest of us can understand what they’re talking about. Still, I couldn’t help wondering whether there are a few things we should be looking for when we’re taking in those presentations. So I decided to ask a researcher.
Dr. Diane Knight is a soil science professor at the University of Saskatchewan. She conducts both small-plot and field-scale research, mostly related to the livestock industry.
Researchers from institutions such as Agriculture and Agri-Food Canada and the universities should be approaching their research without bias, Knight said.
“Whether we get good results or bad results should not matter. We’re just reporting on what we get,” she says.
But, in life, we are going to run across people lacking scruples every now and again, and even the most scrupulous person makes mistakes. Shortly after I talked to Knight, I ran across a Globe and Mail article about Dr. Jonathan Pruitt, a McMaster University researcher facing scrutiny over his previously published, peer-reviewed studies on spider behaviour. From what I gathered, researchers scared the arachnids and then timed how long it took them to emerge from their hidey-holes. Some are saying Pruitt fudged the numbers, but it’s also possible he made a colossal mistake, from what I read.
But picking up on research flaws during a farm-show presentation would be tough, to say the least. Knight says she looks at the statistical setup of a study to evaluate whether it was done correctly. Researchers don’t go into those details at producer talks, with good reason. That level of detail is more likely to clear the room than anything.
Nor should producers get too bogged down in the scientific details, Knight adds. Instead, focus on the presenter’s take-home message and whether it applies to your situation. Hopefully the presenter has “distilled it down into something that’s useful.”
Still, Knight had a couple of suggestions on evaluating research. Say, for example, you’re looking at a new product such as an inoculant. Make sure you understand how the comparison was done between crops that received the inoculant and those that didn’t. Knight says a vague remark about getting an increase isn’t good enough. Researchers should have a properly set-up control (which doesn’t include the inoculant) and a treatment (which includes the inoculant).
Other than the inoculant application, everything should be the same between the control and treatment (time period, soil type, crop type, etc.). For example, you don’t want to apply the inoculant to all the acres one year, and compare those plants to the previous year’s crop, as the weather and many other factors will skew the results. Same goes for a treatment and control on different soils.
Knight says producers don’t need to worry about whether the results are means or averages, but it’s not a bad idea to pay attention to the numbers.
“You want to have some indication of how variable measurements are,” she says. If the mean is 2,000, and the data ranges from 1,000 to 3,000, that is very different than if it varies from 1,999 to 2,001.
“If there’s really tight variability, then that value has more meaning.” In other words, you should put less stock in highly variable data than more consistent results.
This, coincidentally, relates back to the spider-scaring research. When researchers looked at the time it took for each spider to re-emerge, too many of those measured times were identical numbers to be realistic. The implication is that the numbers were changed, perhaps to produce extremely consistent results (or, again, that there was some sort of huge data management mistake).
Knight says that she never tries to give the impression that she knows more than producers. Producers know their own operations best. That also means it’s hard for her to answer questions about a producer’s operation because she isn’t conducting research on that specific operation. She suggests producers “take a more general look” at the research message and then apply it to their operations.
Knight has two livestock-related research projects on the go right now. One is a large project tracking the entire greenhouse gas cycle in a grazed pasture system. Knight is focusing on how different plants affect greenhouse gas emissions from the soil. Dr. Bart Lardner, who is also part of the research team, suggested seeding legumes into the brome grass pasture to boost its nutritional value. Knight says the cicer milkvetch looks like it might be reducing methane emissions from the cattle grazing it. This project is looking at the entire system, which is challenging, but “it’s really fun,” she says.
She is also involved in a project at the Livestock and Forage Centre of Excellence looking at precision manure applications. Dr. Jeff Schoneau is focusing on the agronomy, while Knight and Dr. Richard Farrell are looking at the greenhouse gas emissions over the entire watershed. Researchers are also tracking manure and micro-organisms moving into the water body. Dr. Terry Fonstad, who specializes in environmental engineering, is also part of the project.
Both of these projects involve people from different disciplines working together to collect data on a problem. I mentioned that it seems like there’s more and more of this type of research these days, and Knight confirmed that there’s a big push for these types of multi-disciplinary projects. It sounds like we can expect more of these types of projects in the future.
At any rate, I hope I’ve given you something to think about next time you’re at a conference… or dealing with a spider.