High-Fidelity Through-Occlusion 3D Reconstruction via Millimeter-Wave Surface Normal Estimation

The past few years have witnessed growing interest in millimeter wave (mmWave) based reconstruction in the mobile community [3, 5, 7, 14]. Unlike classical vision-based imaging systems, which are limited to line-of-sight, these mmWave-based systems can operate in through-occlusion scenarios, enabling them to sense objects in closed boxes and beneath clutter. This is because mmWave signals can traverse through many everyday occlusions (e.g., cardboard, fabric, etc.) [1, 11], and reflect off objects behind these occlusions, allowing them to produce images of the occluded objects. This capability, combined with the recent emergence of low-cost commercial mmWave radars, has the potential to enable many promising applications. For example, pick-and-place robots can leverage through-occlusion reconstructions to find and manipulate hidden objects, such as those beneath clutter or within a closed box. Similarly, Augmented Reality (AR) devices could leverage them to perceive occluded objects and display them to the user, truly augmenting our human perception. Smart home devices can use them for through-occlusion gesture recognition, to enable non-verbal commands even when users are hidden from view.