GOAT: GO to Any Thing

在家庭和仓库等部署场景中，移动机器人被期望能够自主导航并执行人类操作者直观理解的任务，而且能够连续工作很长时间。我们提出了“GO To Any Thing”（GOAT），这是一个通用导航系统，具有以下三个关键特征：a）多模态：它可以通过类别标签、目标图像和语言描述来实现目标；b）终身学习：它从过去的经验中受益，适应同一环境；c）平台无关：它可以快速部署到具有不同实体的机器人上。GOAT通过模块化系统设计和不断增强的实例感知语义记忆实现，该记忆可以跟踪来自不同视角的对象外观以及类别级语义。这使得GOAT能够区分同一类别的不同实例，以便导航到由图像和语言描述指定的目标。在涵盖675个目标，跨越9个不同家庭的90多个小时的实验比较中，GOAT实现了83%的总体成功率，比之前的方法和消融实验提高了32%（绝对改善）。GOAT在环境中积累经验后表现更好，从第一个目标的60%成功率提高到探索后的90%成功率。此外，我们还展示了GOAT可以轻松应用于下游任务，如拾取和放置以及社交导航。

The paper proposes a universal navigation system for mobile robots to autonomously navigate and execute tasks in homes and warehouses based on multimodal inputs. The goal is to improve the success rate and platform adaptability of existing navigation systems.

The key idea of the proposed system, GO To Any Thing (GOAT), is a modular design and an instance-aware semantic memory that can handle goals specified via category labels, target images, and language descriptions. The system benefits from its past experience in the same environment and can be quickly deployed on robots with different embodiments.

The experiments conducted in 9 different homes show that GOAT achieves an overall success rate of 83%, surpassing previous methods and ablations by 32% (absolute improvement). GOAT improves with experience in the environment, from a 60% success rate at the first goal to a 90% success after exploration. The system can also be applied to downstream tasks such as pick and place and social navigation. The paper provides detailed descriptions of the system architecture, memory module, and experimental setup. The authors also release the code and dataset for future research.

Related work includes previous navigation systems such as SLAM-based and learning-based methods. The paper also compares GOAT with other multimodal navigation systems and highlights the advantages of the proposed system. Some related papers include 'Visual Semantic Navigation using Scene Priors' and 'Multimodal Dense Video Captioning'.