DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models

J Wynn, D Turmukhambetov
[Niantic]

DiffusioNeRF: 基于去噪扩散模型的神经辐射场正则化

要点:

  1. DiffusioNeRF 用去噪扩散模型(DDM)作为场景几何和颜色的先验，在训练期间对神经辐射场(NeRF)进行正则化；
  2. DDM 在合成 Hypersim 数据集的 RGBD 图块上进行训练，用于预测颜色和深度图块的联合概率分布的对数梯度；
  3. 通过对 LLFF 和 DTU 数据集的评估，正则化方案提高了新视图合成和 3D 重建的性能；
  4. 所提框架是通用的，可用于正则化 NeRF 的其他方面或用梯度下降优化的其他任务，如自监督单目深度估计或自监督立体匹配。

一句话总结:
DiffusioNeRF 用去噪扩散模型(DDM)对神经辐射场(NeRF)进行正则化，以改善 3D 重建和新视图合成。

To alleviate this problem we learn a prior over scene geometry and color, using a denoising diffusion model (DDM). Our DDM is trained on RGBD patches of the synthetic Hypersim dataset and can be used to predict the gradient of the logarithm of a joint probability distribution of color and depth patches. We show that, during NeRF training, these gradients of logarithms of RGBD patch priors serve to regularize geometry and color for a scene. During NeRF training, random RGBD patches are rendered and the estimated gradients of the log-likelihood are backpropagated to the color and density fields. Evaluations on LLFF, the most relevant dataset, show that our learned prior achieves improved quality in the reconstructed geometry and improved generalization to novel views. Evaluations on DTU show improved reconstruction quality among NeRF methods.

https://arxiv.org/abs/2302.12231
图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除