Seeing in the Dark with Ambient Guidance

A low-light image taken in a dark scene usually suffers from severe distortions, which does not accurately characterize the ambient lighting. Long exposure is an accustomed way to capture more supplementary light and alleviate the degradation, but sometimes it induces other distortions, e.g. blurriness. To address this issue, we propose a new paradigm that introduces additional captured ambient guidance, i.e. a long-exposure image to steer the low-light enhancement. In practice, this long-exposure image can be obtained conveniently, but usually suffers from blurriness and misalignment. To effectively extract and fuse information from degraded and misaligned low-light and guidance image pairs, we propose a Long Exposure Compensation Network (LECNet). Adaptive Band Regression is introduced to disentangle the image into multi-scale representations and coarse-to-fine aggregate them with an attention mechanism. For stable image-guidance registration and artifact suppression, we propose a Bounded Cross-domain Deformable Alignment to warp the guidance based on extracted feature pyramids step by step. To integrate knowledge about the degradation into our LECNet for better fidelity, a dual learned back projection is enforced between the predicted result and the paired inputs in illumination and texture detail consistency, serving the model training for both offline training and online sample-adaptive finetuning. For training and evaluation of this new paradigm, we build a dataset with both synthetic and real-captured image triplets of long/short exposure pairs and extra blurry guidance. The experimental evaluation demonstrates the significance of our new paradigm, as well as the superiority of our LECNet and its usability in the real world.