Long-Prompt Zero-Shot TTS: Benchmarking Prosody Reproduction Beyond Voice Cloning

Dekun Chen ,

Renjie Xu ,

Yingda Shen ,

Qinke Ni ,

Zhizheng Wu

GenAI TTS

2026年10月14日

零样本文本到语音（TTS）通常以一段较短的语音作为提示条件，该提示可传达说话人的音色，但仅能提供有限的、关于其韵律模式（如语调、节奏、重音等）的证据。本文提出“长提示零样本TTS”（Long-Prompt Zero-Shot TTS）这一新任务：在推理阶段使用较长的语音提示，从而在不针对目标说话人进行微调的前提下，准确复现其个性化的韵律特征。为此，我们构建了LP-TTS-Bench——一个双语基准测试集，涵盖37位说话人，总计包含63.95小时的提示语音和2,728条预留的测试语句。除语音可懂度与音色相似度外，LP-TTS-Bench还专门评估语音的时长节奏（timing）、重音分布（prominence）、语调轮廓（intonation），以及基于模型计算的多项韵律指标，从而实现对韵律复现能力的多维度、精细化评测。作为基线参考系统，我们设计了LPTTS模型：其分层式“长提示编码器”（Long-Prompt Encoder）可将长语音提示高效压缩为固定数量的连续韵律向量。实验结果表明，相较于短提示基线系统，LPTTS在多个韵律维度上均取得显著提升；整体性能随提示语音时长增加而持续改善，增益效应在提示时长约10分钟时趋于明显，更长的提示则带来边际收益递减，性能趋于饱和。

Zero-shot text-to-speech (TTS) typically conditions on a short speech prompt, which conveys timbre but provides only limited evidence of a speaker's prosodic patterns. We formulate Long-Prompt Zero-Shot TTS, a task that uses long speech prompts at inference time to reproduce speaker-specific prosody without target-speaker fine-tuning. We introduce LP-TTS-Bench, a bilingual benchmark of 37 speakers with 63.95 prompt hours and 2,728 held-out test utterances. Beyond intelligibility and timbre similarity, LP-TTS-Bench evaluates timing, prominence, intonation, and model-based prosody metrics, enabling multidimensional evaluation of prosody reproduction. As a reference system, we develop LPTTS, whose hierarchical Long-Prompt Encoder compresses long speech prompts into a fixed number of continuous prosody vectors. Experiments show improvements in several prosodic dimensions over a short-prompt baseline, with overall gains as prompt duration increases up to approximately 10 minutes and saturation at longer durations.

许愿