Recursive Principal Component Analysis-Based Data Outlier Detection and Sensor Data Aggregation in IoT Systems

Internet of Things (IoT) is emerging as the underlying technology of our connected society, which enables many advanced applications. In IoT-enabled applications, information of application surroundings is gathered by networked sensors, especially wireless sensors due to their advantage of infrastructure-free deployment. However, the pervasive deployment of wireless sensor nodes generate massive amount of sensor data, and data outliers are frequently incurred due to the dynamic nature of wireless channels. As operation of IoT systems relies on sensor data, data redundancy and data outliers could significantly reduce the effectiveness of IoT applications or even mislead systems into unsafe conditions. In this paper, a cluster-based data analysis framework is proposed using recursive principal component analysis (R-PCA), which can aggregate the redundant data and detect the outliers in the meantime. More specifically, at a cluster head, spatially correlated sensor data collected from cluster members are aggregated by extracting the principal components (PCs), and potential data outliers are determined by the abnormal squared prediction error score, which is defined as the square of residual value after extraction of PCs. With R-PCA, the parameters of PCA model can be recursively updated to adapt to the changes in IoT systems. Cluster-based data analysis framework also releases the computational and processing burdens on sensor nodes. Practical databases-based simulations have confirmed that the proposed framework efficiently aggregates the correlated sensor data with high recovery accuracy. The data outlier detection accuracy is also improved by the proposed method compared to other existing algorithms.