How Large Language Models Shape Programming Skill Development Beyond Task Completion

Large language models (LLMs) are changing how students approach problem solving, code interpretation, and software development. Successful task completion with AI assistance, however, does not necessarily indicate conceptual understanding of underlying programming principles, making it difficult to determine how programming skills develop over time. This study examines whether students’ critical engagement, collaborative learning practices, and exploratory use of LLMs are associated with self-reported programming competence, coding practices, and longer-term knowledge retention. Survey data from 189 students with varying levels of LLM use in educational and coding-related contexts were analyzed using partial least squares structural equation modeling (PLS-SEM). The findings suggest that students perceive LLMs as more educationally valuable when they actively question, reinterpret, and adapt AI-generated responses rather than accept them as final answers. Students who interacted more reflectively with AI outputs reported stronger perceived competence and more deliberate attention to code organization, readability, and maintainability. Collaborative use corresponded to broader development of programming abilities, whereas creative experimentation was more closely related to stylistic refinement and perceived benefits for longer-term retention. Active engagement with AI-generated material may therefore promote analytical reasoning and deeper conceptual involvement instead of merely accelerating code production. By moving beyond technology adoption and productivity-oriented perspectives, the study highlights the role of learner agency in shaping LLM-supported programming education.