ROSIE 系统利用文本到图像模型扩大了机器人的学习能力，提高了鲁棒性，且不需要额外的真实世界数据。
[RO] Scaling Robot Learning with Semantically Imagined Experience
T Yu, T Xiao, A Stone, J Tompson, A Brohan, S Wang, J Singh, C Tan, D M, J Peralta, B Ichter, K Hausman, F Xia
[Google]
用语义想象经验扩大机器人学习规模
要点:
-
ROSIE 使用文本到图像模型来增强机器人学习数据集，而无需额外的真实世界数据； -
增强的内容包括有语义的背景、新任务和干扰； -
在 ROSIE 增强的数据上训练的策略可以解决未见过的任务，并且对干扰和背景更加鲁棒； -
ROSIE 还提高了机器人学习中成功检测的鲁棒性，特别是在分布外的情况下。
Recent advances in robot learning have shown promise in enabling robots to perform a variety of manipulation tasks and generalize to novel scenarios. One of the key contributing factors to this progress is the scale of robot data used to train the models. To obtain large-scale datasets, prior approaches have relied on either demonstrations requiring high human involvement or engineering-heavy autonomous data collection schemes, both of which are challenging to scale. To mitigate this issue, we propose an alternative route and leverage text-to-image foundation models widely used in computer vision and natural language processing to obtain meaningful data for robot learning without requiring additional robot data. We term our method Robot Learning with Semantically Imagened Experience (ROSIE). Specifically, we make use of the state of the art text-to-image diffusion models and perform aggressive data augmentation on top of our existing robotic manipulation datasets via inpainting various unseen objects for manipulation, backgrounds, and distractors with text guidance. Through extensive real-world experiments, we show that manipulation policies trained on data augmented this way are able to solve completely unseen tasks with new objects and can behave more robustly w.r.t. novel distractors. In addition, we find that we can improve the robustness and generalization of high-level robot learning tasks such as success detection through training with the diffusion-based data augmentation. The project's website and videos can be found at this http URL
https://arxiv.org/abs/2302.11550
内容中包含的图片若涉及版权问题，请及时与我们联系删除
评论
沙发等你来抢