3DSF-MixNet: Mixer-Based Symmetric Scene Flow Estimation From 3D Point Clouds
Shuaijun Wang, Rui Gao, Ruihua Han, Qi Hao
Abstract
The scene flow estimation aims at accurately achieving the motion of 3D points, imposing challenges like mis-registration, object occlusions, and non-uniform upsampling. This paper introduces a scene flow estimation framework featuring a unified scene flow estimator, a symmetric cost volume approach, and a geometric/semantic feature based upsampling strategy. The novelty of this work is threefold: (1) developing a novel progressive framework which integrates the cost volume module and scene flow estimator, enhancing scene flow estimation; (2) developing a symmetric inter-frame correlation feature extraction method through cost volume estimation using MLP-Mixer operations; (3) developing an upsampling strategy based on both the semantic and geometric feature similarities between sparse and dense samples. Experimental results show that our method outperforms state-of-the-art baseline methods, especially in scenarios involving challenging conditions, the improvements of our method achieving at most <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{0.109}\, {\text {m}}/0.089\,m/0.091\,m$</tex-math></inline-formula> in EPE3D, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$54.23\%/53.67\%/74.1\%$</tex-math></inline-formula> in AS, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$32.75\%/21.87\%/40.25\%$</tex-math></inline-formula> in AR, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$70.98\%/58.06\%/43.56\%$</tex-math></inline-formula> in outliers, when tested on FlyingThings3D ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathrm{FT3D_{S}}$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathrm{FT3D_{H}}$</tex-math></inline-formula> ) and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathrm{KITTI_{H}}$</tex-math></inline-formula> datasets, respectively.
BibTeX
@inproceedings{ral2024_3dsfmixnetmixerb,
title = {3DSF-MixNet: Mixer-Based Symmetric Scene Flow Estimation From 3D Point Clouds},
author = {Shuaijun Wang and Rui Gao and Ruihua Han and Qi Hao},
booktitle = {RA-L 2024},
year = {2024}
}