Revealing insights: divergent and convergent tendencies in virtual design creativity groups using machine learning versus human coding

Evaluating ideation processes in collaborative design remains a central challenge in design creativity research, particularly in virtual group settings where large volumes of textual data are generated. Efficiently measuring and identifying creative contributions is essential for effective communication and collaboration. This paper presents two related studies investigating similarities and differences between AI-based approaches and human coders in analyzing text from virtual design-focused meeting transcripts. In Study 1, topic modeling was employed to assess the depth and breadth of topics discussed by virtual groups. At the category level, the approach recovered 10 of the 11 categories identified by human coders (90.9%), suggesting strong alignment in thematic structure. Study 2 uses this validated approach to examine how task structure influences design processes and creative outcomes. Results show that task structure did not significantly impact the number of topics explored or ideas generated, but unstructured groups selected a higher proportion of good-quality ideas than structured groups. In addition to validating topic modeling for analyzing topic coverage in collaborative design meetings, this study offers theoretical insights into design ideation across structured and unstructured virtual sessions, with practical implications for organizations seeking to enhance creativity while saving time and reducing reliance on human input.