End-to-end IoT sensor data simulation and predictive analysis: framework implementation and experimental evaluation

Abstract The rapid expansion of the Internet of Things (IoT) has led to an exponential increase in sensor-generated data, creating challenges for efficient data management and transmission. To address these challenges, SHiELD offers a comprehensive sensor data simulation platform that leverages heuristic techniques such as aggregation, compression, and filtering to streamline data flow without compromising data fidelity. The platform incorporates a suite of advanced predictive models—including ARIMA, LSTM, and Transformer architectures, to accurately forecast sensor behavior and trends. Additionally, SHiELD features fault injection capabilities to evaluate system robustness under adverse conditions. It produces detailed reliability assessments based on metrics evaluating time-series similarity, recovery performance, and transmission quality. Validation experiments, including real-world data acquisition using Arduino-based sensor interfaces and processing on embedded and server platforms, demonstrate that SHiELD’s heuristics can reduce data volume by 8.3% to 13.5% (averaging 9.4%) and lower packet transmission counts by as much as 82.5%. The predictive models integrated within the system achieve strong performance, with F1-scores reaching up to 0.93 and ROC AUC values up to 0.97 for top-performing architectures such as the Transformer and Prophet. Overall, SHiELD serves as an integrated framework for simulating, predicting, and assessing the reliability of IoT sensor data streams.