Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
May 14, 2026Open Access

Novel Problem Generation for Programming Pedagogy in the LLM Era: A Non-Leakage Methodology

View Full Paper
Ask AI
Bookmark
Share

Authors

TLThat Le

Discussion

Loading...

Member takes

Overview

Novel methodology creates non-leakage programming assessment items, enhancing learning measurement without bias.

Key Points

  • The goal is to develop assessment items for programming that measure learners' skills rather than their access to advanced models.
  • Synthesis of training-set contamination and LLM literature to inform methodology.
  • Development of a four-rule non-leakage criterion set for problem-generation.
  • Application of techniques for parametric variation, unique-context grounding, and compositional novelty.
  • Introduced a structured workflow for generating programming problems that mitigates leakage risks.
  • Proposed a design recommendation outlining the percentage distribution of techniques in problem-set creation.
  • Acknowledge that novelty in programming problems is an evolving challenge as training data and models advance.

Cite This Study

That Le (2026) studied this question.

synapsesocial.com/papers/6a056668a550a87e60a1e637https://doi.org/10.5281/zenodo.20131526
View Full Paper
Ask AI
Bookmark
Share