가상 프로덕션에서의 시각 언어 모델을 이용한 각본 기반 카메라 촬영 자동화

LED Wall-based virtual production has emerged, integrating real-time background rendering with physical filming environments. However, camera automation for interpreting natural-language scripts according to cinematic intent in virtual production remains challenging. We propose a VLM-based script-driven camera automation system for virtual production. The proposed system generates a structured shot plan from a given natural-language script using an LLM, selects and tracks the target subject, and controls a robotic camera system according to the planned camera movement. Moreover, when the executed result does not sufficiently reflect the script’s intention, a VLM observes the result video and updates the shot plan. Through three script-based demonstration scenarios in a virtual production environment with camera tracking and a robotic camera system, the results demonstrate that the proposed system can automate camera control while reflecting the cinematic intent described in a natural-language script.