ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation

AK's Picks Multimodal Intelligence Other MMI GenAI Diffusion CV HPMR
2026年07月09日
在交互式应用中实时生成逼真的人体三维运动,对于动画制作、仿真模拟以及类人机器人控制而言至关重要。尽管近期的离线运动生成方法可通过文本指令与运动学约束实现精确控制,但其推理速度尚无法满足交互式场景的需求;而现有的在线生成方法虽能实现实时合成,却往往在可控性方面有所妥协,或因上下文窗口有限,难以准确理解复杂的文本语义、亦难以有效处理长时程目标。本文提出 ARDY——一种流式生成框架,旨在弥合上述差距,支持基于在线文本提示与灵活运动学约束的高保真人体运动生成。ARDY 采用一种混合表征方式,将显式的根节点运动特征与隐式的全身姿态嵌入相结合,在确保运动轨迹精准控制的同时,兼顾生成建模的高效性。我们设计了一种两阶段自回归式 Transformer 去噪器,其具备可变长度的历史上下文感知能力,并支持对灵活设定、长时程的运动学约束进行条件建模。通过在大规模动作捕捉数据集上进行训练,并直接以真实动作姿态中采样的文本标签与运动学约束作为条件输入,ARDY 原生地习得了可控生成能力,从而天然支持在线文本提示输入与灵活的长时程目标规划。我们在 HumanML3D 基准数据集及大规模、高保真度的 Bones Rigplay 数据集上开展了全面评估,结果表明 ARDY 不仅生成运动质量优异,且对各类运动学约束具有高度遵循性,充分验证了我们核心架构设计的有效性。最后,我们通过一个交互式演示系统展示了该方法的实际适用性:用户可动态输入文本指令、施加多样化的关键帧姿态约束、执行路径跟踪任务,并借助鼠标与键盘实现交互式步态控制。补充视频结果、源代码及预训练模型均已开源,详见 https://research.nvidia.com/labs/sil/projects/ardy/。
Generating realistic 3D human motions in real-time within interactive applications is key for animation, simulation, and humanoid robotics. While recent offline motion generation approaches offer precise control via text and kinematic constraints, they lack the inference speed required for interactive settings. Conversely, existing online methods enable real-time synthesis but often sacrifice controllability or struggle with complex text semantics and long-horizon goals due to limited context windows. In this work, we introduce ARDY, a streaming generation framework that bridges this gap by enabling high-fidelity motion generation controllable via online text prompts and flexible kinematic constraints. ARDY employs a hybrid representation that combines explicit root features with a latent body embedding, balancing precise trajectory control with efficient generative learning. We propose a two-stage autoregressive transformer denoiser that features variable history context and supports conditioning on flexible, long-horizon kinematic constraints. By training on a large-scale motion capture dataset and being directly conditioned on text labels and kinematic constraints sampled from ground truth poses, ARDY natively learns controllable generation that supports online prompting and flexible long-horizon goals. Extensive evaluations on the HumanML3D benchmark and the large-scale, high-fidelity Bones Rigplay dataset demonstrate ARDY's high motion quality and constraint adherence, validating the efficacy of our key architectural decisions. Finally, we demonstrate the method's practical versatility through an interactive demo featuring dynamic text control, diverse keyframe pose constraints, path following, and interactive locomotion control via mouse and keyboard. Supplementary video results, code, and model releases can be found at https://research.nvidia.com/labs/sil/projects/ardy/.
许愿