CNCToolDQN: a deep reinforcement-learning framework for anomaly detection in CNC tool monitoring with tool load and age data

Monitoring tool conditions in CNC manufacturing is crucial for ensuring product quality and process efficiency. While supervised tool monitoring requires costly labeled data, and unsupervised anomaly detection often depends on rigid statistical assumptions, both approaches face limitations in real-world applications. We propose CNCToolDQN, an original deep reinforcement learning framework tailored for CNC tool monitoring. It formulates anomaly detection as a sequential decision process, where the agent learns to detect tool wear and faults by exploring tool load time-series patterns and receiving rewards for identifying expert-defined potential and unlabeled anomalies. CNCToolDQN integrates domain knowledge (e.g., tool load and age) to enhance learning and reduce reliance on labeled data. Furthermore, we introduce a practical Tool health score, derived from anomaly scores, to estimate the remaining useful life of tools and enable prescriptive maintenance decisions. Evaluated on real-world datasets from a South Korean automotive parts manufacturer and a public benchmark milling dataset, CNCToolDQN outperforms state-of-the-art AD models. This study contributes a novel, label-efficient RL approach to tool condition monitoring, designed to align with the realities of industrial CNC operations. Code : https://github.com/cnctooldqnupload/CNCToolDQN.