This technical note presents FT03 v0.3, a minimal controlled diagnostic on the relation between source selection, evidence relevance, and claim authorization in two small instruction models. The study uses synthetic source-claim settings in which models must select a source and a claim from candidates already provided in the interface. The diagnostic separates three operational levels: source validity, evidence relevance, and claim authorization. Across 48 base items and 192 balanced variants, the two tested models selected valid and evidence-relevant sources while retaining substantial Evidential Illegitimacy Rates (EIR): 69/192 for Qwen2.5-1.5B-Instruct and 94/192 for SmolLM2-1.7B-Instruct. The purpose of the note is deliberately narrow. FT03 is not an end-to-end RAG benchmark, not a general citation-auditing framework, and not a claim about frontier models. It isolates a compact controlled failure mode: source selection does not necessarily guarantee claim authorization. The accompanying reproducibility package supports scoring-level reproducibility from included raw model outputs. It contains the dataset, prompts, raw outputs, stored results, summary files, audit code, recomputation code, README files, and manifest needed to verify the reported scoring results. The package does not claim end-to-end model-inference reproducibility and does not rerun the two models.
Danilo Tavella (Thu,) studied this question.