LS-Decomposition for Robust Recovery of Sensory Big Data

The emerging Internet of Things (IoT) systems are fueling an exponential explosion of sensory data. The major challenge to effective implementation of IoT systems is the presence of massive missing data entries, measurement noise, and anomaly readings, which motivates us to investigate the robust recovery of sensory big data. In this paper, we propose an LS-decomposition approach that decomposes a sensory reading matrix as the superposition of a Low-rank matrix and a Sparse anomaly matrix. First, based on data sets from three representative real-world IoT projects, i.e., the IntelLab project (indoor environment), the GreenOrbs project (mountain environment), and the NBDC-CTD project (ocean environment), we observe that anomaly readings are ubiquitous and cannot be ignored. Second, we prove that the convex surrogate of the LS-decomposition problem guarantees bounded recovery error under proper conditions. Third, we propose an accelerated proximal gradient algorithm that converges to the optimal solution at a rate that is inversely proportional to the square of the number of iterations. Evaluations on the above three data sets show that the proposed scheme achieves (relative) recovery error ≤ 0.05 for missing data rate ≤ 50 percent and almost exact recovery for missing data rate ≤ 40 percent, while previous methods have (relative) recovery error 0.04 ~0.15 even at only 10 percent missing data rate.