Semantic generalisable multimodal AI generated content detection via mixture of orthogonal decomposition and product manifold disentanglement

The generalisability of AI-Generated Content (AIGC, e.g., images and videos) detection can be decomposed into two tightly coupled subspaces: a semantic subspace and a forgery subspace. State-of-the-art AIGC detectors primarily focus on enhancing generalisability within the forgery subspace, typically neglecting the influences of the semantic subspace. Moreover, most existing AIGC detection methods consolidate all forged images and videos produced by heterogeneous deepfake generation techniques into a single “fake” class. This design choice overlooks the distinctive characteristics and substantial variability associated with different deepfake generation techniques. To address these limitations, we propose a novel Semantic Generalisable Deepfake Detection (SGDFD) framework that integrates a sparsely gated Mixture of ORthogonal Decomposition (MoORD) architecture. The parameter-efficient fine-tuning strategy based on orthogonal decomposition enhances the generalisation capability of the SGDFD model in AIGC detection, while the MoORD architecture further improves its semantic generalisability. In addition, the SGDFD model learns a product-manifold representation that decomposes the monolithic “fake” class into multiple sub-classes, each corresponding to a distinct deepfake generation method. Extensive experiments conducted on 10 real-world AIGC detection datasets under 3 evaluation protocols demonstrate that the proposed SGDFD model achieves state-of-the-art generalisation performance in AIGC detection across both known and unknown semantic categories.