Semantic characterization of ideation: a computational framework using vector embedding and clustering

The demand for innovation in product design requires prolific and effective ideation. Conversational AI (CAI) systems that use Large Language Models (LLMs), such as GPT (Generative Pretrained Transformer), can augment human creativity, providing numerous ideas. However, the assessment of large sets of ideas remains a challenge due to the reliance on expert human judgements, which have limitations of fatigue, bias, and consistency. This paper introduces a method for automated analysis of large sets of ideas generated by CAI systems and/or humans. The method works by converting textual descriptions of ideas to high-dimensional vector embeddings and then analysing their pattern using mathematical techniques such as UMAP, DBSCAN, and PCA. Samples from the UMAP and DBSCAN clusters of similar ideas were assessed by thirty expert designers through an elaborate survey. The clustering is validated by a strong agreement with that of human judgment. Using PCA, the analysis was then extended to characterise the idea space in terms of the dispersion and distribution of vector embeddings. The effectiveness of the ideation exercise was measured in terms of two new objective metrics known as idea sparsity and cluster sparsity. This enables novice designers to effectively select representative ideas from a large pool.