Fringe Projection Profilometry with Lightweight Transformer-CNN Network for High-dynamic-range Scenes

Abstract 3D reconstruction of high dynamic range (HDR) scenes using a minimal number of images is a critical challenge for fringe projection profilometry (FPP). Deep learning has demonstrated significant potential, but the previous approaches never to consider a balance between accuracy and speed, resulting in a slow inference latency for networks. A lightweight U-shaped Transformer-CNN architecture is proposed that integrates dual attention mechanisms (channel and pixel attention) that enables the network to focus more attentions on HDR regions and an Efficient Transformer layer establish a longer dependence that enabling compensation for the HDR regions by utilizing their surrounding pixels' phase information. To address the data scarcity challenge, the network is first pre-trained on a synthetic dataset and subsequently fine-tuned using the real-world dataset that contains 720 HDR scenes. Experiments have proved that the proposed network achieving a phase error as low as 5.32×10^(-4) rad with only contains 0.55 million parameters and a real-time processing speed of 16.56 FPS on an RTX 4060 GPU. Compared with other networks, the proposed network provides an effective phase recovery performance with faster inference speed for HDR scene reconstruction, which is crucial for actual industry applications.