Qwen3-TTS Technical Report

GenAI TTS
本报告介绍了Qwen3-TTS系列模型——一套先进的多语言、可控性强、鲁棒性高且支持流式合成的文本转语音(TTS)模型家族。Qwen3-TTS具备业界领先的3秒语音克隆能力,以及基于自然语言描述的语音控制功能,既可生成完全新颖的虚拟音色,亦可对输出语音进行细粒度调控。该模型在涵盖10种语言、总计逾500万小时的海量语音数据上完成训练;其采用双路语言模型(dual-track LM)架构实现低延迟实时语音合成,并配备两种专用语音分词器(speech tokenizers): 1)Qwen-TTS-Tokenizer-25Hz 是一种单码本编解码器,侧重于语义信息建模,可与Qwen-Audio模型无缝集成,并借助分块式扩散变换器(block-wise DiT)实现流式波形重建; 2)Qwen-TTS-Tokenizer-12Hz 则通过其12.5 Hz采样率、16层多码本结构及轻量级因果卷积网络(causal ConvNet),实现了极致的码率压缩与超低延迟流式传输,首包响应时间低至97毫秒。 大量实验表明,Qwen3-TTS 在多项客观与主观评测基准上均达到当前最优水平(例如:TTS多语言测试集、InstructTTSEval评测集,以及我们自建的长文本语音合成测试集)。为促进学术界与工业界的研究与应用,我们已将上述两类分词器及全部模型以Apache 2.0开源许可证形式向公众免费发布。
In this report, we present the Qwen3-TTS series, a family of advanced multilingual, controllable, robust, and streaming text-to-speech models. Qwen3-TTS supports state-of-the-art 3-second voice cloning and description-based control, allowing both the creation of entirely novel voices and fine-grained manipulation over the output speech. Trained on over 5 million hours of speech data spanning 10 languages, Qwen3-TTS adopts a dual-track LM architecture for real-time synthesis, coupled with two speech tokenizers: 1) Qwen-TTS-Tokenizer-25Hz is a single-codebook codec emphasizing semantic content, which offers seamlessly integration with Qwen-Audio and enables streaming waveform reconstruction via a block-wise DiT. 2) Qwen-TTS-Tokenizer-12Hz achieves extreme bitrate reduction and ultra-low-latency streaming, enabling immediate first-packet emission ($97\,\mathrm{ms}$) through its 12.5 Hz, 16-layer multi-codebook design and a lightweight causal ConvNet. Extensive experiments indicate state-of-the-art performance across diverse objective and subjective benchmark (e.g., TTS multilingual test set, InstructTTSEval, and our long speech test set). To facilitate community research and development, we release both tokenizers and models under the Apache 2.0 license.
许愿