Semantic and network analysis of design protocols to reveal design framing phenomena

This paper introduces a novel approach to analysing design protocols using a combination of methods. It describes an approach using a synthesis of concept extraction (using an LLM), semantic analysis (using word vectors and conceptual clustering), and network analysis (following graph construction). It suggests that the resulting measures are useful for studying design framing and for aiding qualitative analysis. It demonstrates this technique with data from a study of 17 designers addressing two design problems. The method enables the comparison of designers working on the same problem as well as the study of individual designers’ use of concepts over time during a think-aloud study.