Does GenAI Perform Better than Students and Novice Software Developers in Agile/Scrum Requirements Engineering Tasks?

In Agile/Scrum software development, the product backlog is a key artefact specifying software requirements. However, when starting work in Agile/Scrum settings, many junior software developers struggle with requirements specification, even when detailed requirements elicitation notes are provided. We analysed product backlogs created by 211 student teams to identify common mistake types. On this basis, we elaborated a set of 17 typical mistake types and the corresponding checklist to avoid them. We also analysed the teams’ performance with respect to the identified mistake types in order to determine the most common challenges faced by students. The elaborated checklist and the results of the performance analysis may be useful to educators, students, and junior developers involved in requirements specification. We also analysed the current capabilities of GenAI tools and compared them with the performance of the student teams in similar settings where a product backlog has to be specified based on the detailed notes created within the requirements elicitation process. We conducted experiments with GPT-4.0 Turbo, GPT-6.1 Sol Medium, Gemini 2.0 Flash, Gemini 3.5 Flash-Lite-High, and Val, a version of GPT-4.1 mini refined with university-specific safety requirements. For older models, we present the results from a single run per prompt. For more recent models (GPT-6.1 Sol Medium and Gemini 3.5 Flash-Lite-High), we present the results from three runs per prompt. The experiments allow us to investigate the practical applicability of GenAI tools for drafting the product backlogs and analyse the potential need to adjust settings for the corresponding assignments.