Development of a Personal Color Diagnosis and Virtual Makeup Application Based on Facial Color Data Analysis

Recently, with the growth of the beauty industry, there has been increasing interest in personal colors that stand out in relation to individual beauty. With regard to an offline-based personal color diagnosis, experts conduct the diagnosis based on a sensory evaluation, while an online-based personal color diagnosis utilizes application-based methods that involve analyzing user-captured facial images. However, the online-based personal color diagnosis, widely used for its convenience and low cost, faces challenges related to accuracy due to factors such as lighting conditions and noise that arise when the user captures their facial image. Moreover, existing personal color diagnosis services lack the ability to provide a virtual makeup experience that closely resembles reality. This study aims to achieve consistent online personal color diagnosis results by proposing an algorithm that analyzes user-captured facial images in various color spaces and extracts representative skin colors. To ensure an accurate personal color analysis, potential hindrances to personal color diagnoses were eliminated using a YOLOv5-based object-recognition deep-learning model. Additionally, a clustering-based data analysis algorithm was employed to derive representative facial colors. Based on these representative facial colors, the study diagnosed personal color types and ultimately provided users with a virtual makeup function that enhances their aesthetic experience. The virtual makeup feature uses the BeautyGAN generative deep-learning model to show the most harmonious makeup styles for each user, enabling them to actively select makeup items that best suit their preferences.