Transformative Energy-Efficient Edge-Optimised Multimodal Deep Learning Framework for Pest Management and Severity Analysis in Tea Plants

Recurrent infestations reduce yield and quality in Indian tea cultivation. Early visual symptoms under field conditions are often obscured by low light, high humidity, and leaf occlusion, limiting the reliability of conventional vision-based classifiers to controlled settings. Diverse field environments further degrade performance and constrain deployment on standard edge hardware. The multimodal, edge-optimised framework integrates RGB pest imagery with real-time environmental data for robust on-site monitoring . A hybrid CNN–Transformer backbone captures fine pest-specific textures and long-range contextual cues, while an attention-driven fusion layer adaptively incorporates temperature, humidity, and illumination signals. A key contribution is the custom dual-mode edge hardware , integrating dedicated CNN and Transformer accelerators with sensor co-processors. Hierarchical on-chip SRAM buffers reduce memory energy by 200 ×, supported by energy-aware scheduling. The system enables offline solar-powered inference in connectivity-limited settings and cloud-assisted synchronisation for periodic model updates. The dataset comprises 1,520 field-collected pest images , augmented to 7,600 samples across five classes . Environmental conditioning preserves 85.5% accuracy under combined perturbations, whereas unimodal inference degrades to 77.9% (approximately 15% error reduction). Hardware–algorithm co-design confines inference latency to 25 ms with 0.12 J energy per inference. Energy consumption remains 60% lower than Jetson Nano deployment. The formulation demonstrates that deployment-aware multimodal conditioning stabilises agricultural edge inference under energy, latency, and environmental constraints.