Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces

触觉和视觉相辅相成，相互增强我们理解世界的能力。从研究的角度来看，将触觉和视觉结合起来的问题尚未得到充分探索，因此提出了一种新的方法——触觉信息辅助的三维几何重建和新视角合成。该方法将触觉数据（局部深度图）与多视角视觉数据相结合，通过优化三维高斯基元来准确建模接触点处的物体几何形状。通过创建一个在接触位置降低透射率的框架，我们实现了精细的表面重建，确保深度图均匀平滑。由于现有的方法往往无法准确重建非兰伯特表面（如光亮或反射表面）的高光，因此触觉在考虑这些表面时特别有用。通过结合视觉和触觉感知，我们使用比以前的方法更少的图像实现了更准确的几何重建。我们对具有光泽和反射表面的物体进行了评估，并证明了我们的方法的有效性，提供了重建质量的显着改进。

Tactile-Informed 3DGS: Incorporating Touch to Surface Reconstruction and Novel View Synthesis

The paper proposes a novel approach that combines touch data with multi-view vision data to achieve surface reconstruction and novel view synthesis, using 3D Gaussian primitives to accurately model the object's geometry at points of contact and decreasing transmittance at touch locations for refined surface reconstruction.

The approach offers significant improvements in reconstruction quality for non-Lambertian objects with glossy and reflective surfaces, using fewer images than prior methods. The evaluation is conducted on various objects, and the effectiveness of the approach is demonstrated. No open-source code is provided.

Recent related studies include 'DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation' and 'NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis'.