The power of simulating experiments

Abstract

Addressing complex ecological research questions often requires complex empirical experiments. However, due to the logistic constraints of empirical studies there is a trade-off between the complexity of experimental designs and sample size. Here, we explore if the simulation of complex ecological experiments including stochasticity-induced variation can aid in alleviating the sample size limitation of empirical studies. One area where sample size limitations constrain empirical approaches is in studies of the above- and belowground controls of trophic structure. Based on a rule- and individual-based simulation model on the effect of above- and belowground herbivores and their enemies on plant biomass, we evaluate the reliability of biomass estimates, the probability of experimental failure in terms of missing values, and the statistical power of biomass comparisons for a range of sample sizes. As expected, we observed superior performance of setups with sample sizes typical of simulations (n = 1000) as compared to empirical experiments (n = 10). At low sample sizes, simulated standard errors were smaller than expected from statistical theory, indicating that stochastic simulation models may be required in those cases where it is not possible to perform pilot studies for determining sample sizes. To avoid experimental failure, a sample size of n = 30 was required. In conclusion, we propose that the standard tool box of any ecologist should comprise a combination of simulation and empirical approaches to benefit from the realism of empirical experiments as well as the statistical power of simulations.