Soft labelling for semantic segmentation: Bringing coherence to label down-sampling

Roberto Alcover-Couso, Marcos Escudero-Vinolo, Juan C. SanMiguel

Autonomous University of Madrid

语义分割的软标记：将一致性引入标记下采样

要点：

1、在语义分割中，由于资源有限、根据模型输入调整图像大小或改进数据扩充，通常会执行训练数据下采样。这种下采样通常对图像数据和带注释的标签采用不同的策略。这种差异导致下采样像素和标签之间的失配。因此，随着下采样因子的增加，训练性能显著降低。

2、本文将图像数据和带注释标签的下采样策略结合在一起。为此，本研究提出了一种用于标记下采样的软标记方法，该方法在下采样之前利用了结构内容。因此，将软标签与图像数据完全对齐，以保持采样像素的分布。该建议还为表示不足的语义类生成了更丰富的注释。总之，它允许以较低的分辨率训练具有竞争力的模型。

  1. 探讨了相对熵损失（KL散度）以及交叉熵损失在针对这些软标签的训练模型中的应用。与使用全分辨率图像的最先进方法（见图1中标记为Ours的模型）相比，使用下采样版本的训练图像，最佳结果获得了稍微更好的性能，但需要的计算资源明显更少。
  2. 在实践中，与使用大多数资源的模型相比，即DeepLabV3+的一半分辨率需要（50×24）GB GPU内存[3]，使用使用单个Titan RTX GPU训练的相同架构的框架能够以四分之一的分辨率和不到12GB的内存跑赢该模型（3%），以相同分辨率跑赢（7%）超过24GB的内存和仅2个图像的批量大小。

一句话总结：

实验表明，这种在下采样之前利用结构内容的方案优于其他下采样策略。此外，参考基准测试达到了最先进的性能，但使用的计算资源比其他方法少得多。这一建议使得在资源约束下对语义分割进行竞争性研究成为可能。[机器翻译+人工校对]


In semantic segmentation, training data down-sampling is commonly performed because of limited resources, adapting image size to the model input, or improving data augmentation. This down-sampling typically employs different strategies for the image data and the annotated labels. Such discrepancy leads to mismatches between the down-sampled pixels and labels. Hence, training performance significantly decreases as the down-sampling factor increases. In this paper, we bring together the downsampling strategies for the image data and annotated labels. To that aim, we propose a soft-labeling method for label down-sampling that takes advantage of structural content prior to down-sampling. Thereby, fully aligning softlabels with image data to keep the distribution of the sampled pixels. This proposal also produces richer annotations for under-represented semantic classes. Altogether, it permits training competitive models at lower resolutions. Experiments show that the proposal outperforms other downsampling strategies. Moreover, state of the art performance is achieved for reference benchmarks, but employing significantly less computational resources than other approaches. This proposal enables competitive research for semantic segmentation under resource constraints.

https://arxiv.org/pdf/2302.13961.pdf

内容中包含的图片若涉及版权问题，请及时与我们联系删除