Exploring ChatGPT's Performance in Supporting the Design of Human-Computer Interaction

Background: Large Language Models (LLMs) have significant potential for Human-Computer Interaction (HCI), presenting opportunities to assist several stages of design. However, challenges remain regarding contextual understanding, user-aligned suggestions, and the reliability and transparency of AI-driven decision-making. Purpose: This study examines ChatGPT’s potential in supporting user interface design during requirements gathering, focusing on idea generation and its implications for HCI. Methods: An exploratory study was conducted within the context of a chatbot for detecting depressive signs in university students. ChatGPT-4 generated design artifacts for requirements elicitation using four HCI techniques: Questionnaires, Interviews, Personas, and Scenarios. Neutral prompts were used, without prompt-engineering strategies. The outputs were evaluated through a literature-based comparison with established HCI references Preece et al. [2013], Cooper [2004], Barbosa et al. [2021], and by four academic experts using a 5 point Likert scale across six criteria: pertinence, comprehensiveness, clarity, depth, applicability, and coverage. Results: Findings indicate that generative AI produces well-structured outputs that assist in identifying user needs. Pertinence achieved the highest ratings across all techniques, and experts emphasized ChatGPT’s utility as a “starting point” for design activities. However, limitations were observed regarding depth, comprehensiveness, and coverage. In complex domains such as mental health, outputs tended to be generic, while human-led focus groups captured these aspects more effectively. Conclusion: The results show that generative AI tools like ChatGPT are valuable preliminary resources for supporting early HCI design. However, their limitations point to the continued necessity of human expertise to review AI-generated content in context-sensitive domains. This calls for hybrid workflows where human creativity and judgment are complemented, but not replaced, by AI capabilities.