来自今日爱可可的前沿推介

SegNeRF: 3D Part Segmentation with Neural Radiance Fields

J Zarzar, S Rojas, S Giancola, B Ghanem
[KAUST]

SegNeRF: 基于神经辐射场的3D部件分割

要点:

  1. 提出SegNeRF，一种灵活的3D隐式表示，可以从给定的RGB图像中同时学习外观、几何形状和语义信息；
  2. 尽管在训练期间完全依赖图像监督，但广泛的实验验证了SegNeRF在3D部件分割方面的能力；
  3. SegNeRF是第一个多用途的隐式表示，能在不进行昂贵的测试时优化的情况下，联合重建和分割新对象。

摘要：

神经辐射场(NeRF)的最新进展为新视图合成和3D重建等生成任务提供了令人印象深刻的性能。基于神经辐射场的方法能完全依靠摆放图像来隐式地表示3D世界。然而，在3D部件分割等区分性任务领域，它们很少被探索。本文试图通过提出SegNeRF来弥合这一差距：一种将语义场与通常的辐射场集成在一起的神经场表示。SegNeRF从之前的工作中继承了执行新视图合成和3D重建的能力，并从少量图像实现3D部件分割。在PartNet上的广泛实验表明，SegNeRF能同时从摆放图像中预测几何形状、外观和语义信息，即使是没见过的物体。SegNeRF能从真实场景拍摄的物体的单张图像生成显式3D模型，并进行相应的部件分割。

Recent advances in Neural Radiance Fields (NeRF) boast impressive performances for generative tasks such as novel view synthesis and 3D reconstruction. Methods based on neural radiance fields are able to represent the 3D world implicitly by relying exclusively on posed images. Yet, they have seldom been explored in the realm of discriminative tasks such as 3D part segmentation. In this work, we attempt to bridge that gap by proposing SegNeRF: a neural field representation that integrates a semantic field along with the usual radiance field. SegNeRF inherits from previous works the ability to perform novel view synthesis and 3D reconstruction, and enables 3D part segmentation from a few images. Our extensive experiments on PartNet show that SegNeRF is capable of simultaneously predicting geometry, appearance, and semantic information from posed images, even for unseen objects. The predicted semantic fields allow SegNeRF to achieve an average mIoU of extbf{30.30%} for 2D novel view segmentation, and extbf{37.46%} for 3D part segmentation, boasting competitive performance against point-based methods by using only a few posed images. Additionally, SegNeRF is able to generate an explicit 3D model from a single image of an object taken in the wild, with its corresponding part segmentation.

https://arxiv.org/abs/2211.11215


图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除