Before Calling It a Non-Replication: Instrument Qualification Across LLM Families

When a behavioral effect observed in one large language model is absent on another model family, the result is often described as weak generalization or non-replication. That interpretation assumes that the task remains an interpretable measurement instrument for the new model. This methodological note examines that assumption through a preserved sequence of cross-family qualification attempts following a completed pre-specified fixed-N evidence-weighting study. The original Qwen study found that five explicitly derivative reviews of one source increased retention of an initial claim relative to an equal-count control containing five unrelated memory records (22/32 versus 0/32; paired risk difference 0.6875; Holm-adjusted exact McNemar p = 9.5367e-7). Subsequent Phi and Ministral programs introduced frozen preflight, eligibility, calibration, and redesign gates before permitting confirmatory collection. None reached a valid cross-family confirmatory estimate. A post-publication no-new-data forensic validation independently reproduced the v0.4.3 H1/H2 calculations and found no material data or statistical error, while narrowing claims about preregistration, control matching, and mechanism. The note presents the full history as a case study in instrument qualification and measurement transport across LLM families and proposes a practical reporting distinction among valid replication/non-replication, configuration ineligibility under the instrument, and invalid/inconclusive execution. No new behavioral observations were generated for this manuscript.