How the Representation of Retrieved Context Affects In-Context Prompting for Commit Message Generation

High-quality commit messages are essential software artifacts because they succinctly communicate the intent and scope of code changes, yet large language models (LLMs) often fail to reflect project-specific writing conventions when used in a zero-shot setting without contextual signals. This study investigates not whether retrieval helps, but how the same retrieved example, when represented differently in the prompt, quantitatively changes generation outcomes. We implement a retrieve-then-generate framework where the target commit’s diff is used as a query for BM25 (Best Matching 25)-based sparse retrieval over a commit-level database, and the top-1 similar commit is optionally injected as an example context. We compare a no-context condition (K = 0) against a minimal-context condition (K = 1) under three context representations: Diff-only, Message-only, and Diff+Message pair. Using Qwen-7B on 8000 evaluation samples with a fixed prompt skeleton, deterministic decoding, and identical post-processing across conditions, we observe negligible differences at K = 0 (BLEU-4 1.14, ROUGE-L 7.47–7.48, METEOR 4.88–4.91), establishing a stable baseline. At K = 1, the same top-1 retrieved case yields systematically different metric responses depending on how it is represented (Diff-only, Message-only, or Diff+Message), even under an identical prompt skeleton, deterministic decoding, and identical post-processing. This indicates that “context representation” is not a cosmetic formatting choice but a first-class prompt-design variable in retrieval-augmented in-context learning for commit message generation. Accordingly, practitioners should select the representation based on the intended objective (e.g., lexical/style alignment vs. change-intent grounding), rather than assuming a universally optimal format.