Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Model

目前的化妆迁移方法仅限于简单的化妆风格，因此在实际应用场景中难以应用。本文介绍了一种新颖的基于扩散的化妆迁移方法——稳定化妆（Stable-Makeup），能够稳健地将各种真实世界的化妆效果迁移到用户提供的面部图像上。稳定化妆基于预训练的扩散模型，并利用保留细节的化妆编码器来编码化妆细节。此外，它还采用内容和结构控制模块来保留源图像的内容和结构信息。通过在U-Net中添加新的化妆交叉注意力层，我们能够将详细的化妆效果准确地迁移到源图像的相应位置。经过内容-结构解耦训练后，稳定化妆能够保持源图像的内容和面部结构。此外，我们的方法表现出强大的鲁棒性和通用性，适用于各种任务，如跨域化妆迁移、化妆引导的文本到图像生成等。大量实验证明，我们的方法在现有化妆迁移方法中取得了最先进的结果，并具有广泛的潜在应用前景。

Stable-Makeup: A Diffusion-Based Makeup Transfer Method with Content and Structural Control

The paper proposes a novel diffusion-based makeup transfer method, Stable-Makeup, that can robustly transfer a wide range of real-world makeup onto user-provided faces. The method is based on a pre-trained diffusion model and utilizes a Detail-Preserving (D-P) makeup encoder to encode makeup details. It also employs content and structural control modules to preserve the content and structural information of the source image. With the aid of newly added makeup cross-attention layers in U-Net, the method can accurately transfer the detailed makeup to the corresponding position in the source image. After content-structure decoupling training, Stable-Makeup can maintain content and the facial structure of the source image.

The method exhibits strong robustness and generalizability, making it applicable to various tasks such as cross-domain makeup transfer, makeup-guided text-to-image generation, and more. Extensive experiments have demonstrated that the approach delivers state-of-the-art (SOTA) results among existing makeup transfer methods. The authors provide a detailed description of the makeup transfer process and the architecture of the proposed method. The method is evaluated on several datasets, and the results are compared with other state-of-the-art methods. The authors also provide an ablation study to evaluate the effectiveness of the proposed method's different components.

Related research in this field includes 'BeautyGAN: Instance-level Facial Makeup Transfer with Deep Generative Adversarial Network' and 'Makeup Transfer via Domain Adaptation'.