Agogic: Performance-Timed Music Tokens for LLM-Native Text-to-Symbolic-Music Generation

Junhao Chen ,

Mingjin Chen ,

Jingjia Mao ,

Lin Chen ,

Saining Zhang ,

Minglin Chen ,

Ruocheng Wu ,

Liaoyuan Fan ,

Wenyi Li ,

Mingju Gao ,

Henghaofan Zhang ,

Zhihao Li ,

Hao Zhao ,

Yufei Wang ,

Ruqi Huang

LLM Other LLM GenAI AMUMS

2026年08月04日

文本到音乐语言模型的构建，往往始于一个通常被默认采用的选择：如何对音乐进行分词（tokenization）。这一选择通常与模型主干网络（backbone）、训练数据及训练流程（recipe）紧密耦合，其独立影响迄今从未被单独量化评估。本研究中，我们固定预训练的Qwen3.5模型（参数量从0.8B至27B）、训练数据、计算预算及解码策略，仅在七种不同音乐表征方式之间进行切换，并以每种表征自身所对应的、不依赖模型的性能上限为基准，校准纹理（texture）相关指标。结果呈现出清晰且出人意料的排序规律：决定分布保真度（distributional fidelity）的关键约束变量是音乐表征本身，而非模型规模。将主干模型参数量扩大34倍，弗雷歇音乐距离（Frechet Music Distance, FMD）几乎纹丝不动；而仅更换表征方式，FMD即可降低一半。我们开源了一种面向演奏表现力的高分辨率表征流——PMT（Performance-Resolution Tokenization），其支持10毫秒级时序精度、逐音符力度（per-note velocity）、多声部纹理建模，共含609个符号。在0.8B参数量模型上，PMT取得FMD=159，显著优于各类节拍网格（beat grid）表征（后者FMD为272–286），降幅达1.7–1.8倍，在其他指标上最高可达2.8倍；该差异在非重叠的自助法置信区间（non-overlapping bootstrap CIs）下具有统计显著性。换言之，一个仅0.8B参数的、基于演奏表现力的模型，其分布建模能力已全面超越27B参数的节拍网格模型。这一优势进一步在另一个从零训练的26M参数主干网络、以及另一套独立设计的演奏表现力表征器上复现，表明这是该类表征范式所固有的属性，而非某种偶然出现的词汇表（vocabulary）巧合。它亦非单纯源于更细粒度离散化带来的假象：即便将PMT的起始时间点（onsets）强制对齐至节拍网格的粗粒度分辨率，PMT仍持续领先二者67–129点FMD（n=500）。该效应本质上是分布层面的；至于其是否可听（audible），则属另一独立问题，本研究尚未作答，相关人类感知实验已预先注册（pre-registered）。原生文本描述（caption）的遵循能力整体偏弱，但可被单独增强：一种轻量级的解码时约束机制，在不损害分布建模性能的前提下，使乐器识别F1分数从0.28提升至0.60，调性准确率（Correct-Key）从0.16提升至0.35。我们同步开源全部实验框架（harness）、25个以上训练检查点（checkpoints）、两个高质量语料库（其一包含86,600条文本/标准MIDI/ABC记谱/音频四模态对齐样本；另一包含625万条带文本描述的音乐样本，系当前规模最大的公开音乐文本配对数据集），以及一项“印迹诊断”（imprinting diagnostic）工具：已有公开发表的文本到MIDI系统，在面对不同文本提示时，其生成结果的和弦—时间分布几无变化（在互斥领域中分别为72%与71%），表明其实际输出高度固化于训练数据分布，对输入文本缺乏真正响应。自此，音乐生成领域未来任何关于新表征方式的主张，均可被实证测量，而不再仅止于主观断言。

Text-to-music language models begin with a choice usually made by default: how to tokenize music. Normally entangled with backbone, data, and recipe, its effect has never been measured in isolation. We fix pretrained Qwen3.5 (0.8B-27B), data, budget, and decoding, and swap only the representation across seven tokenizations, anchoring texture metrics to each representation's model-free ceiling. The ordering is clean and surprising: representation, not model size, is the binding variable for distributional fidelity. Scaling the backbone 34x barely moves Frechet Music Distance (FMD), whereas switching representation halves it. PMT, a performance-resolution stream we release (10 ms timing, per-note velocity, multi-track texture; 609 symbols), reaches FMD 159 at 0.8B against 272-286 for beat grids (1.7-1.8x lower, up to 2.8x elsewhere; non-overlapping bootstrap CIs), so a 0.8B performance-resolution model beats a 27B beat grid. It reappears on a 26M from-scratch backbone and a second performance-resolution tokenizer: a property of the class, not one lucky vocabulary. Nor is it a finer-lattice artifact: snapping PMT's onsets to the beat grids' resolution still leaves it 67-129 FMD ahead of both (n=500). The effect is distributional; whether it is audible is a separate question, left open by our probe, with a human study pre-registered. Native caption adherence is weak but separable: a lightweight decode-time constraint doubles instrument-F1 (.28 to .60) and Correct-Key (.16 to .35) at no distributional cost. We release the harness, 25+ checkpoints, two corpora (86.6k aligned across caption/MIDI/ABC/audio; 6.25M captioned, the largest for music), and an imprinting diagnostic: published text-to-MIDI systems reproduce their training distribution near-invariant to the caption (72% vs. 71% chord-time on disjoint domains). The field's next representation claim can now be measured, not asserted.

许愿