Common decoding tests fail when training and test trials share recording blocks

In a survey of studies decoding block-structured variables from neuronal recordings, all 14 drew training and test trials from shared blocks, and 11 treated trials as independent in significance tests. Slow drift can make blocks distinguishable. Here we assigned labels with no information at random to blocks of simulated, Neuropixels, EEG, fMRI and body-worn sensor recordings, and ran common tests. With shared blocks, these tests called most labels decodable, including the training-label rule of the PyMVPA tutorial. Our drop-in block permutation test, permuting all labels by block and refitting the decoder, kept false positives near 5%, as random assignment guarantees. In International Brain Laboratory recordings, the common test called the block side decodable in 75% of regions per session, and 70% on fake block sequences. Pseudosessions (fake sequences from the task's generator) called 10% and 5%. With shared blocks, significance from the common tests does not show decoding. Code, timestamped designs and outputs: https://doi.org/10.5281/zenodo.24008938