Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis

CV 3DGS AI Systems and Hardware MCPKD
最近,一种优化的三维高斯点阵表示法被引入,用于从稀疏图像集合中合成新视角的高保真场景重建。使得这种表示法适用于像网络流媒体和低功耗设备上的渲染等应用,需要显著降低内存消耗并提高渲染效率。我们提出了一种压缩的三维高斯点阵表示法,利用敏感度感知的向量聚类和量化感知的训练来压缩方向颜色和高斯参数。所学习的码本具有低比特率,并在实际场景中实现了高达31倍的压缩率,而视觉质量仅有轻微的降低。我们展示了压缩的点阵表示法可以通过硬件光栅化在轻量级GPU上高效地渲染,其帧率比经过优化的GPU计算管道高出4倍。对多个数据集进行的广泛实验证明了所提出方法的鲁棒性和渲染速度。
Recently, high-fidelity scene reconstruction with an optimized 3D Gaussian splat representation has been introduced for novel view synthesis from sparse image sets. Making such representations suitable for applications like network streaming and rendering on low-power devices requires significantly reduced memory consumption as well as improved rendering efficiency. We propose a compressed 3D Gaussian splat representation that utilizes sensitivity-aware vector clustering with quantization-aware training to compress directional colors and Gaussian parameters. The learned codebooks have low bitrates and achieve a compression rate of up to $31\times$ on real-world scenes with only minimal degradation of visual quality. We demonstrate that the compressed splat representation can be efficiently rendered with hardware rasterization on lightweight GPUs at up to $4\times$ higher framerates than reported via an optimized GPU compute pipeline. Extensive experiments across multiple datasets demonstrate the robustness and rendering speed of the proposed approach.
许愿