INSIGHT: End-to-End Neuro-Symbolic Visual Reinforcement Learning with Language Explanations

神经符号强化学习（NS-RL）已成为一种有前途的可解释决策制定范式，其特点是符号策略的可解释性。对于具有视觉观察的任务，NS-RL需要对状态进行结构化表示，但是由于效率不高，以前的算法无法使用奖励信号来细化结构化状态。可访问性也是一个问题，因为需要广泛的领域知识来解释当前的符号策略。在本文中，我们提出了一个框架，能够同时学习结构化状态和符号策略，其关键思想是通过将视觉基础模型蒸馏成可扩展的感知模块来克服效率瓶颈。此外，我们设计了一个流程，使用大型语言模型来生成简洁易读的语言解释，以解释策略和决策。在对九个Atari任务的实验中，我们的方法显示出比现有NSRL方法更大的性能提升。我们还展示了策略和决策的解释。

Neuro-symbolic reinforcement learning for tasks with visual observations lacks efficiency in refining structured states with reward signals and requires extensive domain knowledge to interpret symbolic policies. This paper aims to solve this problem.

The paper proposes a framework that learns structured states and symbolic policies simultaneously by distilling vision foundation models into a scalable perception module. The approach also uses large language models to generate concise and readable language explanations for policies and decisions.

The approach demonstrates substantial performance gains over existing NSRL methods in experiments on nine Atari tasks. The paper showcases explanations for policies and decisions. The framework is designed to be accessible and interpretable. The paper provides open-source code for the approach.

Related work includes previous neuro-symbolic reinforcement learning methods and other approaches to explainable decision-making in reinforcement learning, such as counterfactual explanations and causal reasoning.