Wizard-of-Oz vs. GPT-4: A Comparative Study of Perceived Social Intelligence in HRI Brainstorming

Human-robot interaction often employs the Wizard-of-Oz (WoZ) paradigm, where a human controls the robot. However, this approach has limitations, such as a lack of autonomy that impedes real-world applications. Large language models (LLMs) can replace WoZ in conversational tasks, such as brainstorming. We propose that, in such application domains, LLM-controlled robots can achieve comparable perceived social intelligence to WoZ-controlled robots. An experiment (n=27, within-subject design) tested this by having participants brainstorm with an LLM- and WoZ-controlled Furhat robot. Bayesian analyses revealed substantial evidence for the null model for perceived social intelligence, social presentation, and social information processing, indicating similar perceptions of social intelligence for WoZ- and LLM-controlled robots. Participants tentatively preferred the LLM-controlled robot, and reliably identified when the robot was WoZ- or LLM-controlled. This study highlights the potential of LLMs to replace the WoZ paradigm and transform HRI in various research and application domains.