Meta | 鲁棒动态辐射场

来自今天的爱可可AI前沿推介

[CV] Robust Dynamic Radiance Fields

Y Liu, C Gao, A Meuleman, H Tseng, A Saraf, C Kim, Y Chuang, J Kopf, J Huang
[Meta & National Taiwan University & KAIST]

鲁棒动态辐射场

要点:

1. 提出一种不需要已知相机姿态作为输入的动态单目视频时空合成算法；
2. 经过精心设计的架构和辅助损失，提高了相机姿态估计和动态辐射场重建的鲁棒性；
3. 在典型的 SfM 系统无法估计相机姿态的挑战性数据集上表现出良好的鲁棒性。

一句话总结:
提出了一种在不需要已知相机姿态的情况下鲁棒重建动态辐射场的方法，通过精心设计的模型和辅助损失有效提升了鲁棒性。

摘要：
动态辐射场重建方法旨在对动态场景的时变结构和外观进行建模。然而，现有方法假设通过运动结构(SfM)算法可以可靠地估计准确的相机姿态。因此，这些方法不可靠，因为 SfM 算法常常在具有高动态对象、纹理质量差的表面和旋转相机运动的挑战性视频中失败或产生错误的姿态。本文通过联合估计静态和动态辐射场以及相机参数(姿态和焦距)来解决该鲁棒性问题。本文通过大量的定量和定性实验来展示该方法的鲁棒性。实验结果显示，与最先进的动态视图合成方法相比具有良好的性能。

Dynamic radiance field reconstruction methods aim to model the time-varying structure and appearance of a dynamic scene. Existing methods, however, assume that accurate camera poses can be reliably estimated by Structure from Motion (SfM) algorithms. These methods, thus, are unreliable as SfM algorithms often fail or produce erroneous poses on challenging videos with highly dynamic objects, poorly textured surfaces, and rotating camera motion. We address this robustness issue by jointly estimating the static and dynamic radiance fields along with the camera parameters (poses and focal length). We demonstrate the robustness of our approach via extensive quantitative and qualitative experiments. Our results show favorable performance over the state-of-the-art dynamic view synthesis methods.

论文链接：https://arxiv.org/abs/2301.02239

内容中包含的图片若涉及版权问题，请及时与我们联系删除