SpheRoPE: Zero-Shot Optimization-Free 360 Panorama Generation with Spherical RoPE

GenAI Diffusion TDIG CV 3DGS NFRF
我们提出了一种零样本、无需训练、也无需优化的框架,通过将球面先验直接注入预训练的扩散Transformer,实现360度全景图像与视频的生成。现有方法要么依赖于在稀缺的全景数据上进行高成本的微调,从而严重限制了泛化能力;要么采用多步优化策略,导致推理延迟过高、难以实用。我们发现,当前主流生成模型在大规模训练过程中已天然具备一定的全景先验知识;但这些自发涌现的能力仍显不足——模型本质上无法满足等距柱状投影(ERP)所施加的严格拓扑约束。为此,我们提出一种零样本、无需优化的方法,在推理阶段即刻解决上述约束问题:球面旋转位置编码(Spherical RoPE)取代了标准的旋转位置嵌入——其低频通道被重新参数化为三维笛卡尔坐标,以原生方式对球面流形进行建模;高频通道则通过谐波量化实现严格的周期性约束。该方法进一步结合互补性的语义畸变分类器自由引导(Semantic Distortion CFG),显式调控几何结构。由此,我们完全避免了模型重训练,同时完整继承了当前最先进生成模型的全部创意表达能力。本方法具有良好的通用性,可适配多种骨干网络及各类360度生成任务。我们在基于文本生成全景图的任务中,分别验证了Flux.1、Flux.2与LTX-Video三种骨干网络上的有效性,性能均达到或接近现有基线水平,且全程无需任何训练。项目主页:https://orhir.github.io/SpheRoPE
We present a zero-shot, training-free and optimization-free framework for generating 360 panoramic images and videos by directly injecting spherical priors into pre-trained diffusion transformers. Existing methods either rely on costly fine-tuning on scarce panoramic data that limits generalization, or leverage multi-step optimization that incurs prohibitive inference latency. We observe that contemporary generative models natively exhibit some panoramic priors from large-scale training. However, these emergent capabilities are insufficient, as the models fundamentally fail to satisfy the rigorous topological constraints imposed by equirectangular projection (ERP). We introduce a zero-shot and optimization-free approach that resolves these constraints at inference time. Spherical RoPE replaces standard rotary position embeddings: low-frequency channels are re-parameterized as 3D Cartesian coordinates to natively encode the spherical manifold, while high-frequency channels are harmonically quantized to enforce exact periodicity. Coupled with complementary Semantic Distortion classifier-free guidance (CFG) that explicitly steers geometry, we avoid retraining and inherit the full creative breadth of state-of-the-art models. Our approach generalizes across diverse backbones and 360 generation modalities. We demonstrate this across text-to-panorama using Flux.1, Flux.2, and LTX-Video backbones, achieving competitive performance against baselines, all while remaining training-free. Project page: https://orhir.github.io/SpheRoPE
许愿