HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model

基于大型语言模型（LLM）的智能代理在各个领域展现出巨大的潜力，作为交互式系统，它们处理环境观测结果以生成目标任务的可执行动作。这些代理的有效性受到它们的记忆机制的显著影响，该机制将历史经验记录为一系列的动作-观测对。我们将记忆分类为两种类型：跨试记忆，跨越多次尝试而累积的记忆，和试内记忆（工作记忆），在单次尝试中积累的记忆。虽然大量研究已经优化了跨试记忆的性能，但通过改进工作记忆利用来增强代理的性能仍未得到充分探索。相反，现有方法通常涉及将整个历史动作-观测对直接输入到LLM中，导致在长期任务中出现冗余。受人类问题解决策略的启发，本文介绍了HiAgent框架，它利用子目标作为记忆块，以分层方式管理LLM代理的工作记忆。具体而言，HiAgent促使LLM在生成可执行动作之前制定子目标，并使LLM主动决定用总结观测替换以前的子目标，仅保留与当前子目标相关的动作-观测对。五个长期任务的实验结果表明，HiAgent的成功率增加了两倍，所需步骤的平均数量减少了3.8。此外，我们的分析表明，HiAgent在各个步骤上始终提高了性能，突显了其稳健性和通用性。项目页面：https://github.com/HiAgent2024/HiAgent。

HiAgent: A Hierarchical Memory Management Framework for Large Language Model-Based Agents

HiAgent leverages subgoals as memory chunks to manage the working memory of LLM-based agents hierarchically, improving performance through efficient working memory utilization.

Experimental results across five long-horizon tasks demonstrate that HiAgent achieves a twofold increase in success rate and reduces the average number of steps required by 3.8. HiAgent prompts LLMs to formulate subgoals before generating executable actions and enables LLMs to decide proactively to replace previous subgoals with summarized observations, retaining only the action-observation pairs relevant to the current subgoal. The project page provides open-source code.

Recent related work includes 'Reinforcement Learning with Unsupervised Auxiliary Tasks' by Jaderberg et al. and 'Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition' by Dietterich.