Six Ways a Closed-Loop Learning Result Can Look Real and Not Be: A Methodological Audit

Closed-loop learning experiments — a substrate driven by a stimulus, its output read by a decoder, its behavior shaped by reward and penalty — are structurally vulnerable to a specific failure mode: a result that looks like learning because something other than the mechanism under test produced the appearance of it. This note documents six independent instances of that failure mode, found and corrected across eight months of work on a single closed-loop organoid-simulation framework (organoid-oi), together with the general shape common to all six and a checklist derived from them. Each instance is reported with the specific numbers involved: a validation suite that reported 47/47 but actually passed 45/47; a "freeze-weights" control that froze the decoder alongside the substrate and so could not attribute anything to either; a three-category task solvable by an untrained network at 100% accuracy; a stimulus-repetition parameter accepted by a function's signature but never referenced in its body, recurring three separate times across unrelated stimulus generators; a passing six-control result later found to be running at ~85 Hz per neuron against a ~0.4-2 Hz physiological target; and a six-control chain that had, before correction, all shared one contaminated upstream measurement — a recorded response captured after reward/penalty current injection rather than before, inflating the apparent effect from a clean +2.6 points (9/15 seeds positive) to a contaminated +17.7 points (14/15 seeds positive). None of the six defects were found by a second team, a reviewer, or an external audit — all were found by the same author who introduced them. We report this as a standalone methodological contribution, independent of any specific positive or negative result it produced, together with a seven-item checklist directly derived from where each defect was actually caught, offered for other researchers building similar closed-loop organoid or neural-interface learning systems. This document accompanies, and is independent of, organoid-oi v4.