Context matters in LLM-driven algorithm design: A controlled study in black-box optimization

Large language models (LLMs) can turn task descriptions into executable algorithms. Coupled with automatic evaluation, this creates an appealing end-user workflow: an engineer can state a computational problem and guide candidate design without first becoming an expert in the relevant algorithmic field. The practical question is what information from earlier generations should be returned to the model. We investigate this question with EASE (Effortless Algorithmic Solution Evolution), using problem-specific black-box optimizer generation as a controlled case study. Twelve strategies vary whether the prompt includes recent algorithms, high-performing algorithms, their combinations, or no earlier outputs. The strategies are evaluated with Gemini 3 Pro, GPT-5.2, and Claude Opus 4.6 on a difficult 30-dimensional GNBG-II benchmark instance. The observed ranking of individual strategies differs across model configurations, but analysis of the underlying context factors reveals a more consistent descriptive pattern. Combining the most recent candidate with the best-performing candidate yields lower mean repetition-best scores than using either component alone for all three evaluated models. In contrast, increasing the amount of recent or performance-selected context does not show a consistent additional benefit across models. The results therefore favor selective context construction over simply maximizing the amount of previous information supplied to the model. The largest improvements also tend to occur during the early generations, while substantial variation remains across repetitions and model configurations. Because the experiment is restricted to a single benchmark instance and five independent repetitions per model–strategy cell, the comparisons are interpreted descriptively and do not establish a universal context strategy or transfer across optimization problems. Within this controlled setting, however, the results indicate that recency and performance-based selection can provide complementary feedback signals, whereas larger contexts are not consistently advantageous. An interactive result explorer is available at https://tbu-ailab.github.io/PaperContext/ ; generated data and source files are available at https://github.com/TBU-AILab/PaperContext .