来自今天的爱可可AI前沿推介

[CV] LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer

N Yu, C Chen, Z Chen, R Meng, G Wu, P Josel, J C Niebles, C Xiong, R Xu
[Salesforce Research]

LayoutDETR: 检测用Transformer也是多模态布局设计器

要点:

  1. 提出LayoutDETR将两个研究领域——布局生成和视觉检测，桥接到一个框架中；
  2. 建立了一个大规模的广告条幅数据集，并对该数据集进行图形布局生成基准测试；
  3. 该方案为图形布局生成设定了新的最高水准。

摘要：
图形布局设计在视觉传播中起着至关重要的作用。然而，手工布局设计要求高，耗时，并且无法扩展到批量生产。虽然生成模型的出现使设计自动化不再高高在上，但定制符合设计师多模意图的设计，即受背景图像约束和前景内容驱动的设计仍然很困难。本文提出LayoutDETR，继承了生成式建模的高质量和现实性，同时将内容感知要求重新表述为检测问题：学会在背景图像中检测布局中多模态元素的合理位置、比例和空间关系。实验证实，该方案为公共基准和新策划的广告横幅数据集的布局生成提供了新的最先进性能。为了实际使用，本文将解决方案构建成一个图形系统，便于用户研究。其设计吸引了比基线更多的主观偏好。

Graphic layout designs play an essential role in visual communication. Yet handcrafting layout designs are skill-demanding, time-consuming, and non-scalable to batch production. Although generative models emerge to make design automation no longer utopian, it remains non-trivial to customize designs that comply with designers' multimodal desires, i.e., constrained by background images and driven by foreground contents. In this study, we propose extit{LayoutDETR} that inherits the high quality and realism from generative modeling, in the meanwhile reformulating content-aware requirements as a detection problem: we learn to detect in a background image the reasonable locations, scales, and spatial relations for multimodal elements in a layout. Experiments validate that our solution yields new state-of-the-art performance for layout generation on public benchmarks and on our newly-curated ads banner dataset. For practical usage, we build our solution into a graphical system that facilitates user studies. We demonstrate that our designs attract more subjective preference than baselines by significant margins. Our code, models, dataset, graphical system, and demos are available at this https URL.

论文链接：https://arxiv.org/abs/2212.09877
图片
图片
图片
图片

内容中包含的图片若涉及版权问题，请及时与我们联系删除