Augmenting Room Light Patterns for Building Automation with Camera Images and Vision Transformer

Building automation is getting smarter with the advancements of machine learning. In this study, we use a Vision Transformer (ViT) and train it with room images and corresponding lighting control output labels to achieve the automation of room lighting. However, appearance of the room changes depending on the lighting patterns, which may cause oscillation in control outputs of the ViT due to the incorrect output caused by the gaps between the trained scene and the actual scene. We propose Lighting Pattern Augmentation (LPA) to create a lighting control system robust to the changes in lighting. LPA has several steps for synthesizing all the possible lighting patterns, including image darkening and mixing steps. We also propose linear mixer and sigmoid mixer for image mixing. According to our evaluation with our prototype room setup, LPA with sigmoid mixer achieved the best accuracy with significantly improving from 70% to 85%, gaining a 15% increase.