NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction

CV NeRF VSER
我们提出了NOVA3R,一种面向无位姿图像集合、以前馈方式实现非像素对齐三维重建的有效方法。与将几何结构严格绑定于每条光线预测结果的像素对齐方法不同,我们的建模方式学习一种全局性、与视角无关的场景表征,从而将三维重建过程与像素对齐解耦。该设计旨在克服像素对齐三维重建中的两大关键局限:(1)不仅能恢复可见区域的点,还能重建不可见区域的点,从而获得完整的场景表征;(2)生成物理上更合理、更符合真实几何规律的结构,在多视角重叠区域中显著减少冗余或重复的几何结构。为实现上述目标,我们引入了一种“场景令牌”(scene-token)机制,用于跨多张无位姿图像聚合场景信息;同时设计了一种基于扩散模型的三维解码器,用以重建完整且非像素对齐的点云。在场景级与物体级数据集上的大量实验表明,NOVA3R在重建精度与完整性两方面均显著优于当前最先进的方法。
We present NOVA3R, an effective approach for non-pixel-aligned 3D reconstruction from a set of unposed images in a feed-forward manner. Unlike pixel-aligned methods that tie geometry to per-ray predictions, our formulation learns a global, view-agnostic scene representation that decouples reconstruction from pixel alignment. This addresses two key limitations in pixel-aligned 3D: (1) it recovers both visible and invisible points with a complete scene representation, and (2) it produces physically plausible geometry with fewer duplicated structures in overlapping regions. To achieve this, we introduce a scene-token mechanism that aggregates information across unposed images and a diffusion-based 3D decoder that reconstructs complete, non-pixel-aligned point clouds. Extensive experiments on both scene-level and object-level datasets demonstrate that NOVA3R outperforms state-of-the-art methods in terms of reconstruction accuracy and completeness.
许愿