Exploring the use of LLMs to evaluate design creativity

ABSTRACT: Creativity is a fundamental aspect of design that can bring us novel and useful products. However, measuring creativity in design can always be challenging as there is a lack of standardized quantification methods and the inherent limitations of mathematical modelling. Previous approaches often rely on human experts to assess design creativity. Still, humans can be subjective and biased in their evaluation procedures. Recent advancements in AI have inspired us to integrate LLMs as evaluators in engineering design. In this study, we utilize LLMs to assess the novelty and usefulness of design ideas. We developed an evaluation procedure and tested it using design samples. Experimental results demonstrate that the proposed method enhances creativity evaluation capabilities across various LLMs and improves the alignment between LLM and human expert assessments.