Reinforcement Learning for Time Series Data with Partially Labeled Anomalies

In recent manufacturing processes, the escalation of sensors usage has led to the continuous collection of large volumes of data. This trends present the challenge of extracting significant patterns and rapidly identifying anomalies within this extensive time-series data. A major obstacle in this area is the scarcity of labeled data for anomaly detection, which limits the effective training of machine learning models. Existing research has mainly concentrated on utilizing the limited anomaly data with supervised learning or investigating unsupervised learning method. This study employs deep reinforcement learning to simultaneously utilize both the abundant unlabeled data and the minimal labeled data for anomaly detection. This paper aims to learn established anomaly patterns and actively explore potential anomalies in unlabeled data, thereby covering both known and undiscovered anomaly patterns. Experiments on multivariate time-series datasets have shown proposed method to outperform existing models in similar situations. The findings of this research are expected to significantly advance effective anomaly detection in manufacturing environments, particularly in contexts where labeled anomaly data is limited.