SynCity 3000: Bootstrapping Scene-Scale 3D Diffusion

AK's Picks GenAI Diffusion TDIG CV 3DGS Other CV
2026年07月06日
我们提出了SynCity 3000,这是一个用于生成全局一致且支持精细化布局控制的三维场景的框架。当前的图像到三维生成模型已具备从单张图像生成复杂三维资产的能力;我们在此基础上进一步拓展该能力,使其适用于整幅场景尺度——具体方法是将生成模型改造为一种卷积算子。为实现这一目标,我们利用新提出的合成数据引擎生成类场景训练数据,并以此对模型进行微调;该合成数据引擎旨在缓解三维场景训练数据严重匮乏的问题。随后,我们将经改造的卷积式生成器应用于由用户提示语生成的整个场景的等轴测(二点透视)图像上,从而生成任意尺寸与复杂度的三维场景。在各类提示语和布局条件下,SynCity 3000均能生成规模宏大、结构连贯、细节丰富的三维场景,有效克服了以往三维场景生成方法存在的诸多局限。
We present SynCity 3000, a framework for generating 3D scenes that are globally coherent while enabling fine-grained layout control. Building on the ability of current image-to-3D generators to produce complex 3D assets from a single image, we extend this capability to the scale of entire scenes by adapting the generator to be applicable as a convolutional operator. We achieve this by fine-tuning the model on scene-like data generated by a new synthetic data engine, which we propose to address the scarcity of 3D scene data for training. The convolutional generator is then applied to a dimetric image of the entire scene, generated from the user prompt, resulting in 3D scenes of arbitrary size and complexity. Across diverse prompts and layouts, SynCity 3000 produces large, coherent, and detailed scenes, addressing the shortcomings of prior approaches to 3D scene generation.
许愿