来自爱可可的今日推介

GH-Feat: Learning Versatile Generative Hierarchical Features from GANs

Y Xu, Y Shen, J Zhu...
[Chinese University of Hong Kong & Ant Research & Hong Kong University of Science and Technology & ...]

GH-Feat: 用GAN学习多功能生成式分层特征

要点:

  1. 提出一种用 GAN 学习分层视觉特征的新方法，即生成式分层特征(GH-Feat)；

  2. GH-Feat 用编码器进行训练，将预训练的 GAN 生成器视为习得损失函数；

  3. GH-Feat具有通用性和可迁移性，可用于各种计算机视觉任务，包括生成和判别性任务，以及仅有少量标注的情况下实现细粒度语义分割。

一句话总结:
GH-Feat是一种用 GAN 学习多功能的分层视觉特征的方法，可用于各种计算机视觉任务，波扩最少化标注的语义分割等。

https://arxiv.org/abs/2301.05315

摘要：
近年来，生成对抗网络(GAN)在合成逼真图像方面取得了巨大成功。GAN生成器学习合成逼真的图像并重现真实数据分布。通过这一点，一个具有多层次语义的分层视觉特征自发出现。本文调研了从图像合成中学到的这种生成式特征在解决广泛的计算机视觉任务方面表现出巨大的潜力，包括生成式任务和更重要的判别任务。首先将预训练的 StyleGAN 生成器视为习得损失函数来训练编码器。编码器产生的视觉特征称为生成式分层特征(GH-Feat)，与分层 GAN 表示高度一致，因此从重建的角度充分描述输入图像。广泛的实验表面 GH-Feat 在一系列应用中的多功能可迁移性，例如图像编辑、图像处理、图像协调、人脸验证、地标检测、布局预测、图像检索等。本文进一步表明，通过适当的空间扩展，GH-Feat也可以仅用少量标注实现细粒度语义分割。

Recent years witness the tremendous success of generative adversarial networks (GANs) in synthesizing photo-realistic images. GAN generator learns to compose realistic images and reproduce the real data distribution. Through that, a hierarchical visual feature with multi-level semantics spontaneously emerges. In this work we investigate that such a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones. We first train an encoder by considering the pretrained StyleGAN generator as a learned loss function. The visual features produced by our encoder, termed as Generative Hierarchical Features (GH-Feat), highly align with the layer-wise GAN representations, and hence describe the input image adequately from the reconstruction perspective. Extensive experiments support the versatile transferability of GH-Feat across a range of applications, such as image editing, image processing, image harmonization, face verification, landmark detection, layout prediction, image retrieval, etc. We further show that, through a proper spatial expansion, our developed GH-Feat can also facilitate fine-grained semantic segmentation using only a few annotations. Both qualitative and quantitative results demonstrate the appealing performance of GH-Feat.


图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除