Cross-modal predictive inconsistency with directional asymmetry modelling for multimodal deepfake detection

Deepfake detection systems intended for real-world deployment face a persistent gap between benchmark accuracy and field reliability: detectors trained to recognise generator-specific artefacts degrade sharply against synthesis methods unseen during training. We present an audio-visual forgery detection system built on a different operating principle: rather than learning what fake content looks like, it models the causal coupling between a speaker’s acoustic output and facial dynamics that authentic recordings share, and flags departures from that coupling as evidence of manipulation. Its core component, Cross-Modal Predictive Inconsistency (CMPI) , trains two lightweight cross-modal predictors on genuine audio-visual pairs only – requiring no generator labels and no paired-fake supervision, a practical advantage against synthesis methods that do not yet exist. Two application-driven extensions improve deployability. Directional Inconsistency Asymmetry Modeling (DIAM) replaces a symmetric error score with a signed asymmetry signal A = ɛ f − ɛ b whose sign identifies which modality was manipulated, giving analysts an interpretable output rather than a bare anomaly score, and achieving 58.6 × real/fake separation versus 8.5 × for the symmetric baseline. Temporal Inconsistency Localization (TIL) attends over per-frame prediction errors to remain reliable when only part of a clip has been manipulated, a common real-world case that clip-level scoring misses. On the FakeAVCeleb benchmark, the full system reaches AUROC 1.000 and 98.62% balanced accuracy. This modality-attribution capability, however, is validated only in-domain: on LAV-DF it attributes the manipulated modality at chance level (49.78% accuracy, AUROC 0.487), so we report it as a validated in-domain mechanism rather than a general zero-shot capability. Because reliability under distribution shift is a deployment-critical property, we go beyond benchmark reporting and directly test transferability to three independent datasets (LAV-DF, DFDM, Celeb-DF), then run six targeted diagnostic experiments – output renormalisation, embedding alignment, calibration-set-size sensitivity, cross-dataset attribution testing, encoder-initialization ablation, and quantitative attention-region analysis, to determine whether the observed generalisation gap is a fixable calibration issue or a deeper representational one. Most rule out simple fixes and localise the gap to the encoder’s learned representations; the encoder-initialization result further shows this is partly attributable to the choice of pretraining, since replacing a deepfake-specific warm-start with generic ImageNet initialization measurably improves cross-domain AUROC at negligible in-domain cost. The system demonstrates a practical, interpretable, and self-diagnosing approach to audio-visual deepfake detection, with a precisely characterised boundary for real-world use.