Data-driven event detection with partial knowledge: A Hidden Structure Semi-Supervised learning method

Enabled by the advancement of data acquisition and data analysis technologies such as sensor networks and machine learning, recently data-driven event detection has shown its advantage in dealing with complex systems especially those with significant stochastic and dynamic behavior. However, existing methods usually adopt supervised learning framework and depend on explicit expert labeling in the learning phase, which is expensive even impractical in many situations. In this work, we propose a new data-driven event detection method, namely Hidden Structure Semi-Supervised Machine (HS 3 M), that only requires partial expert knowledge. The key idea is to combine unlabeled data and partly labeled data in a large margin learning objective to bridge the gap between supervised, semi-supervised learning and learning with hidden structures. Difficulties do arise as the incorporation of extra learning terms makes the problem non-convex. To optimize the learning objective we establish a novel global optimization algorithm, namely Parametric Dual Optimization Procedure (PDOP), by showing that the parametrized dual problem has local explicit solutions and the corresponding optimality is convex in hidden variables. The proposed approach is applied to power distribution network event detection, and the result justifies the effectiveness of both HS 3 M and the new global optimization algorithm.