Generative Panoramic Image Stitching

GenAI Diffusion Inpainting
我们提出了生成式全景图像拼接任务,其目标是生成无缝的全景图,忠实地呈现包含视差效应以及光照、拍摄参数或风格存在显著差异的多张参考图像中的内容。在这一具有挑战性的场景下,传统的图像拼接流程会失效,生成的结果常常出现重影和其他伪影。尽管最近的一些生成模型能够生成与多张参考图像内容一致的扩展区域,但在需要合成全景图中大面积连贯区域的任务中表现不佳。为克服这些局限性,我们提出了一种方法,通过微调基于扩散模型的图像修复模型,使其能够根据多张参考图像保留场景的内容和布局。模型微调完成后,只需一张参考图像即可生成完整的全景图,输出结果无缝且视觉连贯,忠实融合了所有参考图像中的内容。在实际采集的数据集上的评估结果显示,我们的方法在图像质量以及图像结构和场景布局的一致性方面,相较于基线方法有显著提升。
We introduce the task of generative panoramic image stitching, which aims to synthesize seamless panoramas that are faithful to the content of multiple reference images containing parallax effects and strong variations in lighting, camera capture settings, or style. In this challenging setting, traditional image stitching pipelines fail, producing outputs with ghosting and other artifacts. While recent generative models are capable of outpainting content consistent with multiple reference images, they fail when tasked with synthesizing large, coherent regions of a panorama. To address these limitations, we propose a method that fine-tunes a diffusion-based inpainting model to preserve a scene's content and layout based on multiple reference images. Once fine-tuned, the model outpaints a full panorama from a single reference image, producing a seamless and visually coherent result that faithfully integrates content from all reference images. Our approach significantly outperforms baselines for this task in terms of image quality and the consistency of image structure and scene layout when evaluated on captured datasets.
许愿