Event camera guided visual media restoration and 3D reconstruction: a survey

Event camera sensors are bio-inspired sensors that asynchronously capture per-pixel brightness changes and output a stream of events encoding the polarity, location and time of these changes. These systems are witnessing rapid advancements as an emerging field, driven by their low latency, reduced power consumption and ultra-high capture rates. This survey explores the evolution of fusing event stream captured with traditional frame-based capture, highlighting how this synergy significantly benefits various video restoration and 3D reconstruction tasks. This paper systematically reviews major deep learning contributions that use unique data from event camera systems for image/video enhancement and restoration, focusing on temporal improvements, such as frame interpolation and motion deblurring, as well as spatial enhancements, including super-resolution, low-light/high dynamic range processing and artifact reduction. This paper also explores how the 3D reconstruction domain evolves with the advancement of event-driven fusion. Diverse topics are covered, with in-depth discussions on recent works for improving visual quality under challenging conditions. Additionally, the survey compiles a comprehensive list of openly available data sets, enabling reproducible research and benchmarking. By consolidating recent progress and insights, this survey aims to inspire further research into leveraging event camera systems, especially in combination with deep learning, for advanced visual media restoration and enhancement.