Large-scale language models (LLMs) have enabled the creation of autonomous language agents that can solve complex tasks in dynamic environments without task-specific training. However, these agents often face challenges when tasked with broad, high-level goals due to their ambiguous nature and delayed rewards. Frequent retraining of models to adapt to new goals and tasks is impractical, further complicating the problem. Current approaches focus on two types of auxiliary guidance: pre-task decomposition and post-hoc experience summarization. However, these methods have limitations, including a lack of empirical evidence and difficulties in effectively prioritizing strategies. The challenge lies in enabling autonomous language agents to consistently overcome these limitations while still achieving high-level goals without training.
Previous research has explored different ways to mitigate these challenges. Reflexion allows agents to reflect on failures and devise new plans, while Voyager develops a code-based skills library from detailed feedback. Some approaches analyze both failed and successful attempts to summarize causal abstractions. However, the learning gained from feedback is often too general and unsystematic. LLM struggles with long-term, high-level goals in decision-making tasks, which require additional support modules. Decomposition methods such as Decomposed Prompting, OKR-Agent, and ADAPT break down complex tasks into subtasks or use hierarchical agents. However, these approaches often decompose tasks before interacting with the environment and lack well-founded dynamic adjustments. The limitations of existing methods highlight the need for more adaptive and context-aware approaches to achieve high-level goals.
Researchers from Fudan University and the Allen Institute for AI Self Goalis a self-adaptive framework for language agents to achieve high-level goals by utilizing both prior knowledge and feedback from the environment. The main idea is to build a tree of subgoals for a text and let the agent select the appropriate one as a guideline based on the current situation. SELFGOAL has two main modules for manipulating the GOALTREE: a search module that selects the most suitable goal node, and a decomposition module that decomposes the goal node into more specific subgoals. The Act module uses the selected subgoals as a guideline for the LLM to take action. This approach provides precise guidance for high-level goals, adapts to diverse environments, and significantly improves the performance of language agents in both cooperative and competitive scenarios.
SELFGOAL employs a non-parametric learning approach for language agents to achieve high-level goals. It decomposes high-level goals in a top-down hierarchical manner using a tree structure (GOALTREE) for decision guidance. The framework interacts with the environment through three main modules: search, decomposition, and execution. The search module selects from the leaf nodes of the GOALTREE to identify the sub-goals that are most appropriate for the current situation. The decomposition module refines the GOALTREE by decomposing selected sub-goals into more specific goals, using a filtering mechanism to control granularity and avoid redundancy. The execution module then uses these selected sub-goals to update instruction prompts to guide the agent's actions in the environment. This dynamic approach allows SELFGOAL to adapt to changing situations and provide context-appropriate guidance.
SELFGOAL significantly outperforms baseline frameworks in a variety of environments with high-level goals, and the improvement is more pronounced for larger LLMs. Unlike task decomposition methods such as ReAct and ADAPT, which may provide inappropriate or overly broad guidance, and post-hoc experience summarization methods such as Reflexion and CLIN, which may generate overly detailed guidelines, SELFGOAL dynamically adjusts its guidance. For example, in the Public Good Game, SELFGOAL refines subgoals based on observed player behavior, allowing agents to effectively adapt their strategies. The framework also performs well with smaller LLMs due to its logical and structured architecture. In competitive scenarios such as auction competitions, SELFGOAL exhibits clear advantages over baselines, adopting more strategic bidding behaviors to achieve better results.
In this study, the researchers proposed SELFGOAL to enhance LLM's ability to achieve high-level goals across a range of dynamic tasks and environments. By dynamically generating and refining a hierarchical GOALTREE of contextual subgoals based on environmental interactions, SELFGOAL significantly improves agent performance. The method proves effective in both competitive and cooperative scenarios, outperforming baseline approaches. By continuously updating the GOALTREE, agents can navigate complex environments more accurately and adaptively. While SELFGOAL has shown effectiveness even with small models, improved model understanding and summarization capabilities are required to fully realize its potential. Despite this limitation, SELFGOAL represents a major advancement in enabling autonomous language agents to consistently achieve high-level goals without frequent retraining.
Please check Papers and projects. All credit for this research goes to the researchers of this project. Also, don't forget to follow us. twitter.
participate Telegram Channel and LinkedIn GroupsUp.
If you like our work, you will love our Newsletter..
Please join us 44k+ ML Subreddit
Asjad is an Intern Consultant at Marktechpost. He is pursuing a B.Tech in Mechanical Engineering from Indian Institute of Technology Kharagpur. Asjad is an avid advocate of Machine Learning and Deep Learning and is constantly exploring the application of Machine Learning in Healthcare.
