Mathematical reduction of KL divergence to L2 regularization in the loss function of a variational autoencoder

Generative deep learning models have become a standard tool for synthesizing new data, reducing dimensionality, and restoring distorted images. Autoencoders play a prominent role among such architectures. Their primary goal is to form a continuous latent feature space from which the decoder samples information to generate new objects. Traditionally, to ensure this continuity, developers add a penalty term to the loss function. Kullback-Leibler (KL) divergence is typically used for this purpose, forcing the vector distribution to approximate a standard normal distribution. However, calculating this metric requires significant computational resources. Furthermore, it often leads to the loss of fine details and excessive smoothing of the generated images. The subject of this study is an alternative approach to optimizing generative networks. The paper conducts a mathematical and practical analysis of an architecture where the resource-intensive classical metric is replaced by standard quadratic regularization of the model parameters (L2 norm). The research methodology involves the algebraic transformation of quadratic forms and matrix trace operator properties to mathematically reduce the KL divergence formula. Practical verification was performed through the training and comparative analysis of two identical fully connected VAE architectures on the MNIST dataset. The study showed that for a certain class of machine learning tasks, strict classical metrics turn out to be redundant. Mathematical analysis confirmed the internal similarity of the two regularization methods: under normal data distribution conditions, the quadratic weight penalty yields an effect similar to the traditional approach. However, the structure of the latent space changes noticeably. While the standard metric provides the smoothest transitions, the proposed replacement forms a less dense but more structured space. As a result, the algorithm generates clear, standardized objects more accurately. The main practical advantage of the new approach is the reduction in computational load. Model training becomes faster, which is critical when processing large datasets. Ultimately, the choice of the loss function should be based on the final goal of the project. The traditional penalty is optimal for smooth style transfer tasks, whereas the proposed method performs better at generating clear shapes and detecting anomalies.