对比学习在NLP和多模态领域的应用

对比学习的主要思想是相似的样本的表示相近，而不相似的远离。对比学习可以应用于监督和无监督的场景下，并且目前在CV、NLP等领域中取得了较好的性能。

对比学习在NLP领域中的应用

1、A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation

2、CERT：Contrastive Self-supervised Learning for Language Understanding

3、SimCSE: Simple Contrastive Learning of Sentence Embeddings(EMNLP2021)

4、ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding

对比学习在多模态领域中的应用

1、Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (ICML 2021)

2、Align before Fuse: Vision and Language Representation Learning with Momentum Distillation (NeurIPS 2021)

3、VLMO: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts