Balancing performance and cost of LLMs in a multi-agent framework for BIM data retrieval

This study explores strategies for optimizing the use of large language models (LLMs) in Building Information Modeling (BIM) data retrieval. BIM data retrieval plays a crucial role in enhancing the efficiency and effectiveness of building management and construction processes. Utilizing LLMs can significantly improve data accessibility, reduce retrieval time, and support better decision-making. We propose a method to match queries of varying complexity with suitable LLMs within a multi-agent system (MAS) to balance accuracy and computational costs. We evaluated three commonly used LLMs (GPT-3.5 Turbo, GPT-4o, and GPT-4 Turbo) and found that GPT-4o strikes a good balance between performance and cost. By encoding and clustering query statements, we effectively classified query difficulty levels and matched them with appropriate models. Our tests showed that the multi-agent system with the planner mechanism reduced costs by nearly 31% while maintaining the same accuracy compared to systems without the mechanism.