VADER: Visual Affordance Detection and Error Recovery for Multi Robot Human Collaboration

现今的机器人可以利用大型语言模型的丰富世界知识，将简单的行为技能串联成长期任务。然而，由于原始技能故障和动态环境，机器人经常在长期任务中被打断。我们提出了VADER，这是一个计划、执行、检测框架，具有寻求帮助作为一项新技能，使机器人能够在人类或其他机器人的帮助下恢复并完成长期任务。VADER利用视觉问答（VQA）模块来检测视觉可负担性并识别执行错误。然后，它为语言模型规划器（LMP）生成提示，LMP决定何时寻求其他机器人或人类的帮助来从长期任务执行中恢复。我们展示了VADER在两个长期机器人任务中的有效性。我们的试点研究表明，VADER能够通过向另一个机器人寻求帮助来清理桌子，执行复杂的长期任务。我们的用户研究表明，VADER能够通过向人类寻求帮助来清理道路，执行复杂的长期任务。我们收集了19个人对VADER性能与未寻求帮助的机器人性能的反馈。

VADER: A Framework for Recovering from Errors in Long-Horizon Robotic Tasks by Asking for Help

The VADER framework enables robots to recover from errors in long-horizon tasks by asking for help from humans or other robots, using visual question answering modules and a language model planner.

The VADER framework was tested on two long-horizon robotic tasks and showed effectiveness in completing tasks by asking for help. A pilot study and user study were conducted to gather feedback on VADER's performance. The framework leverages visual question answering modules and a language model planner to detect errors and prompt for help.

Related work in this field includes research on long-horizon robotic tasks, visual question answering, and language model planning. Some relevant papers include 'Learning to Ask Questions in Open-domain Visual Question Answering' and 'Learning to Plan with Uncertain Specifications'.