PC²: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction

L Melas-Kyriazi, C Rupprecht, A Vedaldi
[University of Oxford]

PC2: 基于投影条件点云扩散的单图3D重建

要点:

  1. 基于点云的形状表示能实现高度灵活的扩散模型；
  2. 投影调节允许高分辨率的几何形状与输入图像很好地对齐；
  3. 扩散过程的概率性质能实现多个可信的3D点云，可以被过滤以解决单视图 3D 重建问题的不确定性；
  4. 在合成基准上优于之前的方法，在复杂的真实世界数据上实现了很大的质量改进。

一句话总结:
提出一种基于投影条件的条件去噪扩散过程的单图像 3D 重建方法，可获得高分辨率的稀疏几何图形，并能预测图形之外的点颜色。

Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision. In this paper, we propose a novel method for single-image 3D reconstruction which generates a sparse point cloud via a conditional denoising diffusion process. Our method takes as input a single RGB image along with its camera pose and gradually denoises a set of 3D points, whose positions are initially sampled randomly from a three-dimensional Gaussian distribution, into the shape of an object. The key to our method is a geometrically-consistent conditioning process which we call projection conditioning: at each step in the diffusion process, we project local image features onto the partially-denoised point cloud from the given camera pose. This projection conditioning process enables us to generate high-resolution sparse geometries that are well-aligned with the input image, and can additionally be used to predict point colors after shape reconstruction. Moreover, due to the probabilistic nature of the diffusion process, our method is naturally capable of generating multiple different shapes consistent with a single input image. In contrast to prior work, our approach not only performs well on synthetic benchmarks, but also gives large qualitative improvements on complex real-world data.

论文链接：https://arxiv.org/abs/2302.10668
图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除