jina-embeddings-v3: Multilingual Embeddings With Task LoRA

LLM LoRA/Adapter Long Context NLP Rerank MLRCC
我们介绍了jina-embeddings-v3,这是一个具有5.7亿参数的新型文本嵌入模型,它在多语言数据和长上下文检索任务上实现了最先进的性能,支持长达8192个标记的上下文长度。该模型包括一组特定任务的低秩适应器(LoRA),用于生成用于查询-文档检索、聚类、分类和文本匹配的高质量嵌入。此外,Matryoshka表示学习被集成到训练过程中,允许灵活截断嵌入维度而不影响性能。在MTEB基准测试中的评估显示,jina-embeddings-v3在英语任务上优于OpenAI和Cohere的最新专有嵌入,同时在所有多语言任务中相对于multilingual-e5-large-instruct实现了更优异的性能。
We introduce jina-embeddings-v3, a novel text embedding model with 570 million parameters, achieves state-of-the-art performance on multilingual data and long-context retrieval tasks, supporting context lengths of up to 8192 tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA) adapters to generate high-quality embeddings for query-document retrieval, clustering, classification, and text matching. Additionally, Matryoshka Representation Learning is integrated into the training process, allowing flexible truncation of embedding dimensions without compromising performance. Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the latest proprietary embeddings from OpenAI and Cohere on English tasks, while achieving superior performance compared to multilingual-e5-large-instruct across all multilingual tasks.
许愿