Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

Multimodal Intelligence Vision-Language Pre-training VQA GenAI Diffusion TDIG
2026年06月25日
目前,大多数支持视觉理解与图像生成双重能力的统一型大尺度多模态模型(LMM)仍需依赖人工精心构建的后训练监督信号,例如人工标注、偏好标签,或外部训练的奖励模型。我们提出一个问题:是否存在一种统一型LMM,仅利用无标注图像即可自主提升其视觉理解与图像生成这两项能力?为此,我们设计了一种自演化的训练框架,该框架内部包含三个协同角色:一名“提议者”(Proposer),负责生成视觉相关问题;一名“求解者”(Solver),负责回答并评估这些问题;以及一名“生成器”(Generator),负责合成图像。整个训练过程仅依赖模型自身推导出的一致性信号,完全无需人工标注、偏好标签,亦不借助任何针对特定任务训练的外部奖励模型或评判模型。为增强学习稳定性,我们引入了“求解者词元熵”(Solver Token Entropy, STE)——一种基于词元级预测不确定性的连续难度信号;即使在样本级一致性信号变得不可靠时,该信号仍能持续提供有效指导。在图像生成方面,我们设计了一种多尺度内部评估机制,将问答保真度评分与循环一致的图像描述(cycle-consistent captioning)相结合。由此形成一种由求解者中介的耦合关系:更优的视觉理解能力可带来更可靠的生成质量评估,并进而产生更强的内部训练信号。该框架在扩散模型架构的BLIP3o、基于校正流(rectified flow)的BAGEL,以及自回归架构的VARGPT-v1.1三种不同主干网络上,均保持相同的角色划分、奖励逻辑与训练调度策略,仅需调用各主干模型原生的提示(prompting)与生成接口即可。在全部八项视觉理解评测指标上,本方法均稳定优于对应的基础模型。以BAGEL为例,其在MMMU基准上的性能绝对提升达+3.5%,图像生成能力在GenEval评测中亦从82%提升至85%。相关代码与模型已开源发布。
Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a $+3.5\%$ absolute gain on MMMU and improves GenEval image generation performance from $82\%$ to $85\%$. Code and models are publicly released.
许愿