Going beyond mean effect size: Presenting prediction intervals for on-farm network trial analyses

Abstract

The aim of on-farm research is to identify and test a new technology, product or management practice (e.g. more efficient seeding rate, enhanced row spacing, better disease management treatment, etc.) suited to local con- ditions by comparing it to a standard farmer practice across several farmers’ fields. Typically, each trial includes two treatments (new practice vs. standard control practice) replicated at least three times in each field. The statistical analysis of yield data collected in such trials provides growers with useful information about the effectiveness of the tested farming practice on crop productivity and its uncertainty. We used a random-effects model to i) estimate the performance of a treatment compared to a control in individual trials, ii) estimate the overall mean yield response across all trials, iii) compute prediction intervals describing a range of plausible yield response for a new (out-of-sample) field at the trial level, and iv) compute the probability that the tested management practice will be ineffective in a new field. We used frequentist (classical) and Bayesian approaches for data collected in 26 on-farm trial categories managed by the Iowa Soybean Association. Depending on the level of between-trial variability, we found that prediction intervals were 2.2–12.1 times larger than confidence intervals for the estimated mean yield responses for all tested management practices. We conclude that pre- diction intervals should be systematically reported to provide additional information about future trials or ex- periments with associated uncertainties. Nevertheless, prediction intervals should be interpreted with caution when the between-trial variance is small. Using prediction intervals and, when appropriate, the probability of ineffective treatment will prevent farmers from overoptimistic expectations that a significant effect at the overall population level will lead with high certainty to a yield gain on their own farms.