Madhava and Tracer-MED under Adversarial Audit: What Four Benchmarks Show, and What They Do Not

Four benchmarks measure, independently, the Madhava vector-search engine and the Tracer-MED clinical product built on it. The design is adversarial: every claim is tested against a control that would refute it if false — random-context placebo, shuffled multiple-choice alternatives, complete versus truncated relevance ground truth, exact scan versus approximate index. Main findings. (1) The Madhava index sits within 0.2% of the exact scan on the tested corpus (recall 0.2687 versus ceiling 0.2674), with zero Cauchy-Schwarz bound violations across 172,000 issued certificates. (2) The certificate proves exclusion, not relevance: scope is pool_only in 285 of 285 queries, and the same certificate configuration spans acceptance from 0.533 to 0.617 depending on the corpus — the certificate does not move, the quality does. (3) The corpus determines everything else: with patient records (descriptive data) as corpus, retrieved context harms the clinical answer (delta = -0.060, p = 0.029); with clinical knowledge as corpus, it helps (delta = +0.083, p = 0.0041), and gains +0.097 over a random-document placebo (p = 0.0013). Role separation. The article declares, for each experiment, which component was under test — index, product, or corpus — and holds the others fixed. Public correction. An earlier version of this work reported +0.147 where the correct value is +0.083. Section 7 documents the error, its causes (an answer parser that read "The answer is B" as A, and a retrieval query that included the four multiple-choice alternatives), and why the smaller number is the defensible one. Both versions remain available and identified, so third parties can verify which is which. Additional results. Contamination is ruled out by a shuffled-alternatives arm (delta = -0.003, p = 1.00). Accuracy conditional on retrieval subject rises 10.4 points when the retrieved document matches the question's subject, versus 5.2 for random context — the contrast that shows retrieval causes the gain rather than prompt length. A WORM evidence-chain tamper test is included; tampering is detected. Embedder comparison across four models is reported, including that a 0.6B embedding model outperforms its own 4B sibling by 0.042 p@10 at one fifth the latency. Artifacts include benchmark source code, full result JSONs with 1,200 per-item records (including raw LLM output), and the audit document that motivated the corrections. All numbers in the article are traceable to the deposited JSON files.