How Close to the Frontier? Measuring the Fidelity of Deployable Open-weights LLMs to a Frontier Reference for Clinical Phenotype Extraction

Frontier large language models read clinical narratives at near expert level, but hospitals often cannot deploy them: privacy regulation, data-residency rules, and on-premises requirements forbid sending patient text to a cloud service.The deployable alternative is a small open-weights model run locally.We ask a scoped fidelity question: how closely do such models reproduce a frontier reference for clinical phenotype extraction (cohort identification from records, not bedside diagnosis)?On a 500-case subset of PMC-Patients and six guideline-defined cardiometabolic conditions, we place seven self-hostable open-weights models between a keyword floor and a frontier ceiling, the consensus of two frontier models.The strongest reach macro-F1 of about 0.95 against the ceiling; under a realistic deployment prompt, a 14B model matches a 70B (no significant difference), and a three-small-model ensemble significantly outscores the standalone 70B, at far lower hardware cost.Once self-hosted, a case carries no per-call cost, so deployment becomes a hardware decision.We release the benchmark, reference, and code.