Advanced data analytics for on-farm experimentation: methods, challenges, and opportunities for modern agronomic decision making

On-farm experimentation (OFE) has become an important approach for generating agronomic knowledge under real-world farming conditions. Advances in yield monitoring, auto-guidance, and variable-rate application technologies enable spatially explicit experiments at operational scales, but OFE datasets often violate classical statistical assumptions developed for small-plot experimentation. As a result, specialized analytical methods are required. While simple experimental designs and analyses remain valuable for farmer-led learning and pragmatic decision-making, recent advances in statistical and machine-learning (ML) methods have expanded analytical capabilities for OFE. Developments in linear mixed models, Bayesian spatial approaches, and permutation-based methods improve the handling of spatial heterogeneity, uncertainty, and limited replication. In parallel, ML approaches offer flexible modeling of nonlinear and high-dimensional yield responses, with interpretable techniques supporting identification of site-specific yield drivers. Despite these advances, challenges remain related to model generalization, endogeneity in pooled datasets, and representation of temporal variability. Cross-site synthesis, including meta-analysis and Bayesian hierarchical modeling, has emerged as a key strategy for strengthening inference beyond individual trials and supporting regional decision-making. Low-cost sensing technologies, particularly smartphone- and UAV-based yield estimation, are expanding opportunities for OFE in smallholder systems where data scarcity and low mechanization constrain experimentation. Simulation-based approaches, including synthetic datasets, process-based crop models, and hybrid modeling strategies, provide valuable tools for evaluating analytical methods, causal inference, and experimental designs prior to field deployment. Overall, methodological progress is moving OFE toward more robust, scalable, and interpretable analytics, supporting adaptive, data-informed crop management across diverse farming systems.