FLAT: Feedforward Latent Triangle Splatting for Geometrically Accurate Scene Generation

AK's Picks GenAI Diffusion TDIG CV 3DGS
2026年06月23日
仅凭单张图像生成可交互探索的三维场景,既需要强大的生成先验知识,也需要精确、适用于下游任务的几何表征。当前的视频扩散模型能够生成高质量内容,并在潜在空间中隐式地编码多视角几何结构。然而,现有的前馈式潜在场景解码器通常输出体素化的三维高斯分布(3D Gaussians),这类表示缺乏明确定义的表面,因而难以用于仿真模拟或标准图形渲染管线。这一局限促使我们转向解码与表面严格对齐的几何基元——这类基元不仅可直接渲染,而且更接近显式的几何资产。由此,我们提出一个核心问题:能否将经过压缩的视频扩散模型潜在表示,通过单次前向传递,直接映射为显式的表面基元?为此,我们提出了FLAT方法,并首次证明:三角形面片(triangle splats)可直接从视频扩散模型的潜在表示中解码得到。相较于解码三维高斯分布,预测扁平化(flat)基元 notoriously 更具挑战性,因其对基元朝向高度敏感,常常导致梯度流严重恶化。FLAT通过两大关键技术解决该难题:一是采用以光线为中心的旋转参数化方案,用于三角形回归;二是设计了一种新颖的乘积型窗函数(product window function),显著改善了可微分三角形渲染过程中的梯度传播。在标准基准测试中,FLAT在几何精度方面显著优于当前最先进的前馈式基线方法,同时视觉质量仍保持竞争力。我们进一步表明:仅需一个轻量级的测试时精细化步骤(test-time refinement),即可将预测所得的“三角形集合”(triangle soup)转换为完全不透明、可直接导入游戏引擎的三维表示,从而支持实时渲染。此外,我们在完全一致的训练设置下,对三维高斯溅射(3DGS)、二维高斯溅射(2DGS)以及三角形面片溅射(triangle splatting)这三种变体进行了系统对比评估,首次全面分析了前馈式场景生成中不同表征方式的权衡取舍。项目主页详见:https://flat-splat.github.io
Generating explorable 3D scenes from a single image requires strong generative priors and accurate geometric representations suitable for downstream use. Current video diffusion models offer high-quality generation and implicitly encode multi-view geometric structure in latent space. However, existing feedforward latent scene decoders typically output volumetric 3D Gaussians that lack a well-defined surface, limiting their use in simulation or standard graphics pipelines. This motivates decoding surface-aligned primitives that are not only renderable but also closer to explicit geometric assets. We ask whether compressed video diffusion latents can be mapped directly to explicit surface primitives in a single pass. To this end, we introduce FLAT and, for the first time, show that triangle splats can be decoded directly from video diffusion latents. Compared with decoding 3D Gaussians, predicting flat primitives is notoriously more challenging due to high sensitivity to primitive orientations, oftentimes leading to poor gradient flow. FLAT solves with two key ingredients: a ray-centered rotation parameterization for triangle regression and a novel product window function that improves gradient flow during differentiable triangle rendering. On standard benchmarks, FLAT achieves significantly better geometric accuracy while maintaining competitive visual quality compared to state-of-the-art feedforward baselines. We further show that a lightweight test-time refinement step converts the predicted triangle soup into a fully opaque, game-engine-ready representation that supports real-time rendering. By evaluating 3DGS, 2DGS, and triangle splatting variants under an identical training setup, we provide the first systematic analysis of representation tradeoffs in feedforward scene generation. The project page is available at https://flat-splat.github.io
许愿