The role of background variability for interpreting biological relevance and statistical significance in collembola soil field studies

Soil mesofauna field studies conducted for ecological risk assessments of plant protection products (PPPs) require expertise for interpretation. The new statistical Closure Principle Computational Approach Test (CPCAT; introduced for Poisson-distributed count data) is proposed as alternative for established methods. However, the biological relevance of potential effects remains unclear. One aim was to investigate how to assess biological relevance. Biological Control Ranges (BCR) were calculated for each taxon and sampling time point within individual studies and it was compared whether the average abundance of a treatment group is in- or outside of the BCR. High control abundance variability of Collembola resulted in largely varying BCR values between studies. The second aim was to investigate performance of different statistical methods. Statistical analysis revealed that abundance data exhibited overdispersion in the majority of cases, instead of Poisson-distribution. Significant differences were observed in more than 50% of the comparisons using CPCAT, even before test item application happened in the field. Post-application, most statistical significances determined by CPCAT occurred despite the absence of a rate-response relationship and the arithmetic mean abundance values in the assigned treatment plots being within the BCR; thus, they should be considered as false positive results. In contrast, the Dunnett’s test and Ranked Dunnett’s test, either considering normally distributed data or being independent from the data distribution, barely detected a significant difference at pre-application when the abundance average values of assigned treatment plots were within the BCR pre- and post-application. In conclusion, pre-requisites for CPCAT (Poisson distribution, no over- or underdispersion of data) do not apply in the majority of the cases examined here. Holistic interpretation of field data needs to consider the ecological relevance of observations (via comparison with the BCR), presence of a rate-response-relationship, onset, scale, and pattern of the response, instead of solely focusing on statistics.