Guidiff: Rethinking Difference as Auxiliary Guidance for Neural Video Representation

Neural video representations have recently explored using temporal differences as dynamic cues to enhance motion modeling. However, existing difference-based hybrid methods mostly fuse differences as parallel feature streams and rely on feature-level fusion, which limits the potential of difference information. We propose Guidiff, a neural video representation framework that treats temporal differences as auxiliary guidance signals for decoding. We reinterpret temporal differences as low-frequency structural cues to emphasize their guiding role, and incorporate them as auxiliary conditions to steer intra-frame decoding during progressive up-sampling, thereby improving structural alignment. In addition, we introduce a multi-scale fusion mechanism to further integrate features across decoding stages. Experiments on regression, compression, and inpainting demonstrate that Guidiff consistently outperforms prior difference-based methods and achieves competitive performance with state-of-the-art hybrid approaches on Bunny, UVG, and DAVIS.