The Verdict Is a Function of the Slice: Auditing a Published Quantization Superiority Claim

A published comparison is read as deployment guidance, and what that reader needs is that its verdict holds on the files they are about to download. Two moves are supposed to make such a reading trustworthy: fix what will be read before the data exist, and reproduce it on a second inference engine. We price both, auditing whether one published sentence — that AWQ "consistently outperforms" GPTQ "across different model scales (7B-70B)" — transfers to released artifact pairs inside a deployment envelope declared in advance, one consumer-class 12 GB accelerator. Neither returns what it is bought for. Five of the eight downstream tasks we froze did not resolve the 1-point effect we had ourselves declared meaningful, the task we pre-registered among them (1.187--1.221 points of half-width); the second serving stack changed no verdict — a null belonging to our aggregation convention as much as to the engines — while the downstream criterion run on it alone accounts for 55.78% of the full audit's measured GPU seconds, and two criteria on one engine return the full audit's verdict on every claim cell at 40.14% of that cost. What buys an answer is the second criterion. Treating the sentence as an estimand and varying criterion, engine, task and aggregation convention orthogonally — minimum effect, primary task and analysis frozen before any confirmatory data existed — we find two criteria returning opposite conclusions on one released pair at a checkpoint inside the sentence's own stated range, both clearing the frozen effect: on the audited papers' own criterion corpus, perplexity places AWQ behind GPTQ by 0.993% and 0.920% on two serving stacks matched at the kernel class, while LAMBADA — one of the 3 tasks resolving the effect there — places AWQ ahead by 3.136 and 3.272 accuracy points. The direction turns over at two further layers of the instrument, in preliminary readings entering no verdict: on a second evaluation corpus — disjoint from the one calibration recipe either lineage publishes, the others disclosing none — all four perplexity arms change sign, and a tokenizer packaging setting outside either algorithm's specification moves two of them from excluding zero to containing it. Whether such a sentence transfers therefore depends on which slice is read and on what that slice resolves: one slice, including one fixed in advance, decides it in neither direction.