Improvement of the Method for Speaker-Metadata Conditioning in Fake-News Detection

Relevance. The LIAR and LIAR2 benchmarks are among the most widely used datasets for automatic fake-news detection, and the speaker credit-history counts they provide are the main feature that distinguishes metadata-based detectors from plain text classifiers. If the gains these counts produce arise from how the dataset was built rather than from a real signal, then many leaderboards comparisons measure something that cannot exist when a detector is actually deployed, which makes the integrity of these counts an important problem. The object of research is the process of conditioning a transformer text encoder on speaker metadata for fake-news detection on the LIAR-family benchmarks. The subject of the research is the temporal leakage contained in the pre-aggregated credit-history counts and the methods used to remove it before the features reach a model. The purpose of this paper is to improve the method of speaker-metadata conditioning by using credit-history features that satisfy the principle of temporal honesty, and to assess how much of the apparent metadata gain is leakage rather than usable signal. Research methods. A DeBERTa-v3 encoder is conditioned on the metadata through Feature-wise Linear Modulation under three regimes that differ only in how the counts are computed: leaky, as shipped; split-safe, using the standard remove-self correction; and temporal-honest, using only counts from a speaker's earlier statements. Differences in AUC on the shared test set are assessed with the DeLong correlated-AUC test, and a metadata-only logistic regression serves as an architecture-independent control. Research results. Conditioning on the shipped counts raises test AUC-ROC from a text-only baseline of 0.815 to 0.888, but under the temporal-honest cutoff the advantage inverts and falls below the baseline (Δ = −0.042 AUC; DeLong z = −3.91, p = 9×10⁻⁵). The remove-self correction does not remove the leakage and scores highest of the three regimes, while the key drop from split-safe to temporal-honest is large and clear (Δ = +0.137 AUC, z = 19.5). The logistic regression shows the same pattern, which places the effect in the features rather than the model, and the result is confirmed on the original LIAR dataset (raw 0.826 → leakage-safe 0.692 ≈ text baseline 0.698). Conclusions. Most of the apparent speaker-metadata gain on LIAR-family benchmarks is temporal leakage: honest metadata gives no real benefit over text and may even slightly reduce performance. Useful added features must be both orthogonal to the text and honestly available at inference time, so credit-history counts should be recomputed under a temporal cutoff before they are used or compared.