Noise Optimization in Artificial Neural Networks

Artificial neural network (ANN) has been widely used in automation. However, the vulnerability of ANN under certain attacks poses a security threat to critical automation systems. Previous research has shown that adding noise to ANNs can enhance robustness. Nonetheless, striking a balance between robustness and task performance remains challenging, as excessive noise improves robustness but hampers performance, while low noise offers minor robustness improvement. In this work, we propose to learn the distribution of optimal injected noise, which improves the robustness as well as maintains the performance. Specifically, we compute the pathwise stochastic gradient estimate with respect to the standard deviation of the Gaussian noise added to each neuron of the ANN and optimize both the noise distribution and model parameters during training with negligible additional computational cost. In numerical experiments, our proposed method can achieve significant performance improvement on the robustness of several popular ANN structures under both black box and white box attacks. We also evaluate the proposed technique on two automation tasks: the classic reinforcement learning task of the cart pole game and a fault detection problem. Our results showed that the proposed technique outperforms a conventional neural network in terms of performance, robustness, and visual explainability. Note to Practitioners —The robustness of artificial neural networks is a critical consideration in automation applications as real-world data is often subject to unforeseen perturbations from the environment, potentially causing AI systems to behave unpredictably and unstably. For example, object detection is a widely employed AI technique in automation applications. However, current object detection systems are vulnerable to noise perturbation. Even small, imperceptible noise can lead the model to malfunction. Our work focuses on improving the robustness of neural networks. We propose a novel technique that can be added to any layer of existing neural networks to enhance robustness. Extensive experiments conducted in various scenarios have verified the effectiveness of the proposed method in enhancing both performance and robustness.