What question did this study set out to answer?

This research aims to enhance the estimation of 3D motion of objects in videos by introducing a novel deep architecture.

June 1, 2021

RAFT-3D: Scene Flow using Rigid-Motion Embeddings

Key Points

This research aims to enhance the estimation of 3D motion of objects in videos by introducing a novel deep architecture.
Developed RAFT-3D, a new deep learning model tailored for scene flow estimation.
Used rigid-motion embeddings to group pixels into rigid objects.
Implemented Dense-SE3 layer to ensure geometric consistency of pixelwise motions.
Achieved an accuracy improvement on FlyingThings3D, with δ < 0.05 increasing from 34.3% to 83.7%.
On KITTI benchmark, reduced error to 5.77, outperforming the previous best method at 6.31.

Abstract

We address the problem of scene flow: given a pair of stereo or RGB-D video frames, estimate pixelwise 3D motion. We introduce RAFT-3D, a new deep architecture for scene flow. RAFT-3D is based on the RAFT model developed for optical flow but iteratively updates a dense field of pixelwise SE3 motion instead of 2D motion. A key innovation of RAFT-3D is rigid-motion embeddings, which represent a soft grouping of pixels into rigid objects. Integral to rigid-motion embeddings is Dense-SE3, a differentiable layer that enforces geometric consistency of the embeddings. Experiments show that RAFT-3D achieves state-of-the-art performance. On FlyingThings3D, under the twoview evaluation, we improved the best published accuracy (δ < 0.05) from 34.3% to 83.7%. On KITTI, we achieve an error of 5.77, outperforming the best published method (6.31), despite using no object instance supervision.

Perguntar à IA

Bookmark