BaLi-RF: Bandlimited Radiance Fields for Dynamic Scene Modeling

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Anton Van Den Hengel

[Amazon]

BaLi RF：用于动态场景建模的带限辐射场

要点：

1、从单个移动相机推断非刚性动态场景的3D结构是一个欠约束问题。受神经辐射场（NeRFs）在静态场景的照片逼真新颖视图合成中的显著进步的启发，已经提出了动态设置的扩展。这些方法严重依赖于神经先验知识来规范问题。

2、本文重新研究当前的实现可能带来的有害影响，包括有限的表现力、光场和密度场的纠缠以及次优运动定位。作为一种补救措施，研究主张在经典的非刚性运动结构（nrsfm）和NeRF之间架起一座桥梁，使前者的充分研究先验能够约束后者。为此，提出了一个框架，该框架通过将场景公式化为带限高维信号的合成来分解时间和空间。本文在复杂的动态场景中展示了令人信服的结果，这些场景涉及照明、纹理和长距离动态的变化。

一句话总结：

为动态3D场景建模提供了一个新颖的通用框架，该框架允许空间和时间动态的有效分解。这种分解提供了一个平台，可以将精心设计的时空先验（受NRSfM启发）应用于NeRF，从而实现动态场景的高保真新颖视图合成。该框架在包含长距离运动、尺度变化和光/纹理变化的复杂动态场景中展示了令人信服的结果。[机器翻译+人工校对]

Reasoning the 3D structure of a non-rigid dynamic scene from a single moving camera is an under-constrained problem. Inspired by the remarkable progress of neural radiance fields (NeRFs) in photo-realistic novel view synthesis of static scenes, extensions have been proposed for dynamic settings. These methods heavily rely on neural priors in order to regularize the problem. In this work, we take a step back and reinvestigate how current implementations may entail deleterious effects, including limited expressiveness, entanglement of light and density fields, and sub-optimal motion localization. As a remedy, we advocate for a bridge between classic non-rigid-structure-from-motion (\nrsfm) and NeRF, enabling the well-studied priors of the former to constrain the latter. To this end, we propose a framework that factorizes time and space by formulating a scene as a composition of bandlimited, high-dimensional signals. We demonstrate compelling results across complex dynamic scenes that involve changes in lighting, texture and long-range dynamics.

https://arxiv.org/pdf/2302.13543.pdf

内容中包含的图片若涉及版权问题，请及时与我们联系删除