Answers are grounded in this paper and cite their sources.
Evaluates AI performance in programming contests, indicating robust performance stability despite changes.
We present the 2024 edition of the ECN AI Baseline Index (EAII), an accessible benchmark evaluating large language models in competitive programming under contest conditions. The universities division of the 2024 Sapientia–ECN (Efficiency, Challenge, Networking) programming contest included 11 teams and 16 university-track problems and introduced a judging platform restricted to C/C++/Pascal. Using a simple C++ prompting protocol with limited feedback rounds, the AI fully solved 13 of 16 problems (81.25%), partially solved two, and failed on one interactive task. Student teams achieved a median of nine and a maximum of twelve solved problems. The resulting 2024 EAII value is 125%. We provide per-problem and team-level analyses, along with figures summarizing score distributions, problem-specific solve rates, and year-over-year comparisons with 2023. Despite a markedly easier contest and a shift in the AI language channel (C++ versus Python), EAII remains stable relative to 2023, supporting the robustness of the normalization procedure. Interpretation is reserved for the Discussion; this paper focuses on methodology, descriptive analyses, and numerical results.
No takes yet. Share an insight, caveat, or question.
Kátai et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: