Understanding the interaction between human and LLMs-based AI in creative tasks: Insights from UX professionals’ collaboration and communication with ChatGPT

Large Language Model applications (LLMs), such as ChatGPT, have showcased their remarkable capability in creative tasks with exceptionally human-like responses. To effectively integrate LLMs in creativity-related activities, further research is needed to understand how humans interact with LLMs in creative tasks. In this work, we examined human-LLMs interaction in UX designers’ (N = 12) practices of incorporating ChatGPT into their professional design activities in China, from both collaboration and communication perspectives. Through participants’ longitudinal usage logs, diary surveys, and interviews, we found that participants employed different working strategies to interact with ChatGPT, revealing various collaboration and communication features. The collaboration features highlight how humans and LLMs coordinate task organization, covering task distribution, task contextualization, task fine-tuning with examples, and task evolution through iterative improvement of prompts. The communication features focus on the nature of intentions exchanged between humans and LLMs, including proactive communicator, content format, semantic continuity of prompts, and the speech acts, which cover the communicative goals, contexts, and tones. We also observed participants adjusting their interaction patterns in response to evolving design task demands, particularly shifting from personification to instrumentalization, from improvisational prompts to normalized prompts, and adapting strategies to avoid fixed responses. Grounded in these empirical findings and existing frameworks, we developed a framework to describe human-LLMs collaboration and further argue that communication could serve as an effective lens to understand human-AI interaction. Ultimately, we share implications for designing more productive and human-centered LLMs for creative tasks. • Highlight #1: The research offers a comprehensive understanding of communication-based interaction patterns between designers and LLMs-based AI in real-world creative tasks from a communication perspective, particularly in a longitudinal setting. • Highlight #2: The research develops a human-LLM interaction framework for creative tasks, revealing interaction features (e.g., speech acts, task evolution) that enable prompt customization and seamless collaboration between creative-related practitioners (i.e., designers) and LLM-based AI systems. • Highlight #3: The research shares implications for the design of more user-centered LLM AI applications for both creative and general tasks.