智源TALK｜EmbodiedRSI 把有限试错变成机器人真本事，哥伦比亚&普林斯顿大学

哥伦比亚大学与普林斯顿大学的EmbodiedRSI工作，打造了一套自进化智能体外层框架，让机器人自主判断下一步该探索什么，把实体交互的试错结果，直接转化为可复用的代码与操作技能。让 Embodied Agent Harness 自主决定“接下来试什么、往哪个方更向新”—— EmbodiedRSI ，通过维护代码与能技的改进假设，优先选择信价息值高、成本低的实验，再据根实际执行果结推动代码技与能共同更新；同时通过 reward 学习哪些经验得值保留和复用，让每次交都互能更有效指地导后续改进。在冻结层底机器人型模的情况下，RoboCasa365 未见合组任务成功率最从佳基线的 40.1% 提升至 71.3%，零样本迁移到真机，成率功从 46.0% 提升至 71.3%。相同 50 次交互算预下，相随比机探索，成功率从 27.1% 提升至 72.9%。

Robot foundation models provide strong visuomotor control, yet their performance can degrade when object positions or task instructions change. Further improvements often require post-training on substantial robot data, which can be costly to collect through methods such as teleoperation. Agentic harnesses can adapt around the model, but current self-evolving harnesses use robot trials inefficiently when deciding which code and skill changes to pursue. We introduce EmbodiedEvo, a self-evolving agentic harness that autonomously decides where to explore next and turns the resulting physical interaction into improved code and skills. EmbodiedEvo realizes this through a Fast-Slow Dual-System Architecture, in which competing code and skill hypotheses are maintained in a Hypothesis Graph. Value-of-Information Experiment Selection chooses physical experiments that can distinguish these hypotheses. Their outcomes guide Code-Skill Co-Evolution. The Slow System builds Hierarchical Memory, and Reward-Grounded Memory Learning selects effective memory according to their value for later Fast-System improvement. On RoboCasa365, EmbodiedEvo reaches 77.0% overall success and 71.3% on Composite-Unseen, compared with 40.1% for the best baseline. EmbodiedEvo also reaches 86.8% overall success on LIBERO-Pro. Beyond benchmark performance, EmbodiedEvo transfers zero-shot to real-world robot, achieving 71.3% overall success across multiple challenging tasks.

嘉宾介绍：

I am a PhD student at NYU Courant, advised by Yann LeCun. My research focuses on world models, planning, and representation learning, with the goal of building agents that can learn from observation and interact with the world in an open-ended way. I am particularly interested in hierarchical world models and planning across multiple temporal scales, where structure in the model can enable better long-horizon reasoning, more efficient control, and stronger generalization from offline or reward-free data. More broadly, my interests include self-supervised learning for decision making, computer vision, and embodied intelligence. Before my PhD, I worked at AssemblyAI, where I led the development of Conformer-1 and Conformer-2 and built the company’s first real-time speech recognition model. In a past life, I studied and briefly worked in molecular biology and chemistry before finding my way into AI.

10:30-11:00 

点上方预约，开播有提醒