In general, systems-based studies and factorial experiments confront similar issues when determining the number of replications and plot configurations needed for adequate statistical analyses. A few issues, however, are specific to agricultural systems research and can be addressed by the experimental design and sampling strategies.
To start, what constitutes a “control” for a systems experiment? In factorial experiments, scientists often strengthenthe design by including a control plot that does not receive the treatment (for example, in a fertilizer experiment, the zero-nitrogen treatment is the control). In a systems study, however, this type of control does not usually make sense, and there is generally no straightforward way to define a standard control treatment. Some studies do include a “reference” system, which often consists of the “typical” or “conventional” management system, to compare with a number of experimental alternatives. Depending on the experiment, this system may be referred to as “conventional” or “farmer practice.” This is more accurate than referring to the system as a “control,” since for most agricultural systems studies, the “control” can vary depending on the researchers’ perspectives. In experiments comparing agricultural systems to native or unmanaged native systems, the native ecosystems could be considered “controls” compared to agriculture. In general, avoid designating one system as a “control” because that determination is largely subjective in systems experiments and can lead to confusion.
Spatial variability can also be an issue in systems experiments because of the larger plot sizes. To minimize the impact of unexpected spatial variability in on-farm studies, research can be carried out within smaller, defined field plots (Drinkwater et al., 1995; Schipanski et al., 2010). Farmers, who are usually knowledgeable about spatial variability in their fields, can point out areas that behave differently and should be avoided. Within very large fields, smaller research plots can be used to target particular soil types to allow for better control of soil variability across sites. In field station experiments, allocate resources (e.g., time, funds) for a spatial assessment of the site if it is relatively large. A “uniformity trial” can be done by planting a single crop across the entire experimental site and collecting soil samples and plant biomass data on a defined grid. These baseline studies can be conducted for several years. GPS can be used to identify sampling locations, and geostatistical techniques can characterize spatial variability. Cavigelli et al. (2005) present an excellent example of this technique.
When considering how to block an experiment, analyses of spatial variability can be particularly useful. If possible, arrange blocks to reflect soil variability. Preliminary assessment of the research site has the added advantage of providing a framework that can be used for future sampling and to establish a strong baseline. As the experiment progresses, researchers can document changes over time by sampling a subset of points that were characterized during the preliminary assessment.
Because systems experiments tend to focus on longer- term trends and changes over time, establish a well-documented time-zero baseline, particularly for simulated, replicated system studies, to strengthen the ability to make these assessments. Cropping systems experiments in which the site is repeatedly planted to a single crop often begin with two to three baselines or year zeros. Allow multiple years for baseline data collection if the site includes fields with different management histories that have been combined for the experiment. Careful time-zero sample collection and archiving can also greatly expand future research possibilities. For example, consider collecting and archiving air-dried samples and freezing small samples (ideally at –80°C) in case there is a future interest in using molecular or other techniques.