This paper introduces an approach that leverages large language models (LLMs) to convert detailed descriptions of an Operational Design Domain (ODD) into realistic, executable simulation scenarios for testing autonomous vehicles. The method combines model-based and data-driven techniques to decompose ODDs into three key components: environmental, scenery, and dynamic elements. It then applies prompt engineering to generate ScenarioRunner scripts compatible with CARLA. The model-based component guides the LLM using structured prompts and a “Tree of Thoughts” strategy to outline the scenario, while a data-driven refinement process, drawing inspiration from red teaming, enhances the accuracy and robustness of the generated scripts over time. Experimental results show that while static components, such as weather and road layouts, are well captured, dynamic elements like vehicle and pedestrian behavior require further refinement. Overall, this approach not only reduces the manual effort involved in creating simulation scenarios but also identifies key challenges and opportunities for advancing safer and more adaptive autonomous driving systems.
Danso et al. (Sun,) studied this question.