Conditional Returns to Language Adaptation: An Adaptation Ladder Study of Indonesian Short Answer Grading

Indonesian has several adapted language models for automatic short answer grading. Studies disagree about whether specialization is worthwhile; each test one point on the adaptation spectrum, so a losing language-specific model is indistinguishable from useless specialization. An ordinal comparison replaces it: a three-rung ladder of released checkpoints, from no Indonesian adaptation through regional to national continued pretraining, in two families whose rungs share a base model and parameter count. In the Gemma family, the first two rungs are siblings rather than ancestor and descendant, which the analysis carries through. Across 30,960 grading calls on 36 independent questions, with the question as the unit of inference, returns are conditional rather than cumulative. Regional adaptation helps both families, by 0.029 and 0.018 pooled quadratic weighted kappa (QWK). The second step adds little on the weaker base, 0.016, and harms the stronger one, −0.029. That decline is established in one family and directional in the other. Labelled examples drawn from the question being graded halve the remaining advantage, and examples from other questions attenuate it further, from +0.069 to +0.005, a difference of −0.064 [−0.107, −0.022], although they grade less accurately in absolute terms. Spreading the demonstrated score levels reduces it to +0.029. All six models moved in the direction predicted from their initial scoring bias at four shots, and five of six at eight shots. Error decomposition locates the gain in a shift in grading severity, not sharper discrimination, and the shift runs in opposite directions in the two families. A 124-million-parameter Indonesian encoder, included as a deployment reference, reaches 0.825 QWK in the social science domain at 596 MB, within 0.027 of the best 9-billion-parameter configurations. Language adaptation is therefore a conditional recalibration. Its value depends on the base model, on labeled answers already held, and on which score levels they display.