Assessment of the color compatibility of garments for building a recommendation system for an outfit

In this paper, we consider the problem of clothes compatibility for total look recommendation systems by means of deep neural networks. This task has become very popular in recent years, primarily due to the growth of online retail sales of clothing. Unlike the existing solutions, we developed a comprehensive model of clothes compatibility evaluation based on color characteristics as well as on the characteristics of the style. As a rule, neural networks are robust to the color characteristics of an image, but color is an extremely important component in the task of a total look evaluation, so such additional branch with color characteristics is well justified. The proposed model uses both: color embedding obtained from color clustering and histograms, and style embedding as an output tensor of ResNet-50 encoder. The paper shows that color embeddings significantly improve the quality of the total look evaluation. The model was trained on Polyvore dataset, which was pre-processed and cleaned from the items not related to the topic of total look compatibility.