A Mind-Map of Reasoning, Tools, Agents, and Collaboration

Introduction

Large Language Models (LLMs) have rapidly advanced in their ability to not only generate text but also reason and act as autonomous agents. Researchers and industry leaders are exploring how LLMs can move beyond single-turn question-answering towards long-horizon decision-making, tool use, and even coordination with other agents. The goal of these efforts is to create AI systems that can plan, execute, and adapt in complex, real-world environments.

Recent work has converged on a paradigm that marries the raw reasoning power of LLMs with structured frameworks – such as external tools, modular sub-agents, memory systems, and multi-agent cooperation – to overcome the limitations of any one component alone.

This article provides a comprehensive survey of the intersection of LLM-based reasoning, tool use, sub-agent architectures, and multi-agent collaboration. We'll define key modular components of such systems and highlight evaluations on emerging benchmarks that ground these ideas in practical tasks. We'll then map out the current state-of-the-art in combining LLM reasoning with structured scaffolding, citing empirical results that demonstrate progress. Finally, we'll discuss open challenges and gaps – from context length limits and stability issues to reproducibility and coordination difficulties – that are guiding ongoing research in building more robust autonomous LLM agents.

Key Concepts and Components

Agent

In this context, an agent refers to an autonomous system driven by an LLM that perceives inputs (e.g. text descriptions of state, tool outputs) and produces outputs as actions or decisions to achieve a goal. An agent encompasses not just the LLM model, but also the surrounding scaffolding that structures its interaction loop.

This includes prompts, interpreters for the model's outputs, and any mechanism to execute actions in an environment. In other words, an agent = LLM plus the framework around it that allows it to interact with the world.

Chain-of-Thought & "Think" Tokens

Chain-of-Thought (CoT) refers to prompting or training LLMs to generate explicit intermediate reasoning steps ("thoughts") before giving a final answer. These steps can be marked by special tokens or delimiters (sometimes called think tokens, e.g. <think>...</think>) to distinguish the model's internal reasoning from its external outputs.

By encouraging step-by-step reasoning, CoT prompting significantly improves performance on complex tasks like math word problems and logical reasoning. For instance, with CoT a 540B model (PaLM) achieved state-of-the-art accuracy on a math benchmark (GSM8K), even surpassing a fine-tuned GPT-3 model.

More advanced variants like Tree-of-Thoughts go further by exploring multiple reasoning paths in a search tree. Overall, "think" tokens and CoT techniques are ways to make the LLM's reasoning process explicit, which can improve problem-solving and allow the agent (or a human overseer) to inspect and verify the reasoning steps.

Tools

In the agent context, tools are external APIs or functions that the LLM can invoke to extend its capabilities beyond text generation. Tools might include search engines, calculators, knowledge base queries, code interpreters, databases, or even the ability to execute code or control a robot.

By using tools, an LLM agent can retrieve up-to-date information, perform computations, or effect changes in an environment. A classic example is an LLM calling a calculator API to solve arithmetic exactly instead of approximating it.

Integrating tool use has been shown to reduce hallucinations and improve accuracy; for example, the ReAct framework combined reasoning steps with actions (like API calls) and outperformed pure reasoning on question answering by retrieving evidence from Wikipedia.

Tools are usually invoked via a special output format (such as a JSON function call in OpenAI's function-calling paradigm) that the agent's scaffolding recognizes and executes. In essence, tools allow an LLM agent to act on the world or query external knowledge, rather than being limited to its static internal knowledge.

Sub-Agents

A complex agent can be decomposed into sub-agents, which are specialized modules or smaller agents responsible for particular subtasks. These sub-agents may themselves be LLMs with specific roles or other AI modules.

For example, a large task (like "plan and build a software application") might be divided among sub-agents such as a Planner, a Coder, a Tester, etc., each handling part of the workflow. One prominent example is the MetaGPT framework, which instantiates multiple GPT-based agents with roles like Product Manager, Software Architect, and Engineer to collaboratively develop software.

Sub-agents can also be organized hierarchically (one agent calling another as a function) or in parallel as a team. The use of sub-agents follows the principle of modularization: breaking a problem into pieces that can be solved more easily and then integrated. This approach can leverage the strengths of different models (or differently prompted instances of the same model) and impose structure that an end-to-end agent might lack.

Memory

Memory refers to mechanisms that allow an agent to retain and recall information beyond the immediate context window of the LLM. Since standard LLMs have a fixed context length (e.g. 4,000 tokens, 32,000 tokens), they cannot natively remember arbitrarily long histories or large knowledge bases.

To overcome this, agent systems implement memory buffers, databases, or vector stores to store important information and retrieve it as needed. There are generally two types of memory:

1. Short-term memory / working memory: which might be a summary of recent dialogue or state that is kept in the prompt (within context length)
2. Long-term memory: which could be an external repository (like a vector embedding database of past events or documents) that the agent can query when needed

For example, the Generative Agents architecture (by Park et al. 2023) keeps a comprehensive record of an agent's experiences in natural language, and continuously synthesizes higher-level reflections that can be retrieved later to guide behavior.

Another approach is the Reflexion framework, where an agent appends self-critique and important observations to a persistent log and uses that log to improve subsequent attempts at a task. Memory is crucial for long-horizon tasks because it mitigates the forgetting that happens once the token limit is exceeded – the agent can remember key facts or decisions from earlier, either by caching them in the prompt or by searching its external memory.

Context Length

The context length of an LLM is the number of tokens of input and output it can consider at once. It effectively bounds how much information the model can utilize in a single inference.

Many state-of-the-art LLMs today have context lengths ranging from 4k to 16k tokens (for models like GPT-3.5, etc.), with specialized versions going to 100k or more (Anthropic's Claude 2 offers up to ~100k tokens context).

A long context is particularly valuable for agents because they often need to consider lengthy instructions, extended dialogue histories, or large environment observations (for instance, the content of an entire webpage or code repository). In the coding domain, the importance of context length is highlighted by SWE-Bench, where an agent might need to ingest an entire codebase's relevant files to fix a bug.

Indeed, one winning approach on SWE-Bench used a 200k-token context to allow the model to work with a substantial portion of a repository in one go. Despite progress, context length remains a limiting factor – if a task requires more information than the model can attend to (e.g. a very large codebase or a long-running dialogue), the agent must rely on external memory or heuristics to manage the context.

Thus context length is both an enabling feature (the larger it is, the more complex the tasks the agent can handle directly) and a constraint that agent designs must work around.

Role Assignments

In multi-agent or multi-role systems, each agent or module can be given a specific role that defines its behavior or knowledge scope. Role assignment is a way to imbue structure and avoid confusion when multiple LLMs (or one LLM with multiple personas) interact.

For example, in a collaborative setting you might assign one LLM the role of a User (proposing goals or problems) and another the role of an Assistant (providing solutions) – this was done in the CAMEL framework to enable two ChatGPT-based agents to cooperate on a task by role-playing.

In MetaGPT's "AI company" analogy, roles like Product Manager or Engineer are assigned to different agents to mirror a human organizational structure. Even within a single-agent system, role assignment can be implicit via prompt context (e.g. a system message that says "You are an expert travel planner" vs "You are a bash terminal").

Clearly defined roles help by constraining each agent's output style and function, which improves coordination. They essentially partition the cognitive space so that, say, one agent focuses on high-level planning while another focuses on low-level execution. This reduces interference and can introduce expert skills to different sub-parts of a problem.

Coordination Mechanisms

When an agent is composed of multiple sub-agents or when multiple agents work together, there must be a coordination mechanism to manage their interactions. Coordination can be centralized or decentralized.

A central coordinator (which might be another LLM agent or a simple algorithm) could allocate tasks to sub-agents and integrate their results – as seen in HuggingGPT, where a ChatGPT instance first plans tasks, then calls appropriate expert models in sequence, and finally compiles the results.

Alternatively, coordination can emerge through communication protocols among agents: for example, agents might take turns sending messages to each other (a form of multi-agent dialogue) to negotiate and share information. In such cases, rules are needed for turn-taking, message understanding, and conflict resolution.

Some frameworks use a blackboard or shared memory that all agents can read/write to, ensuring that updates from one agent inform the others. Generative Agents in the simulated town used a shared environment (the sandbox world) and a syncing of each agent's memory when they interacted.

Key coordination mechanisms include scheduling (deciding which agent acts when), communication language (natural language vs structured messages), and decision protocols (e.g. majority voting among agent outputs, or a "critic" agent that evaluates proposals from others). Effective coordination is what prevents a multi-agent system from devolving into chaos – it ensures the agents work towards a common goal rather than at cross purposes.

Chain-of-Thought Reasoning for LLM Agents

One of the fundamental advances enabling LLMs to tackle complex reasoning tasks is the use of Chain-of-Thought (CoT) prompting. Chain-of-thought refers to guiding the model to produce a sequence of intermediate reasoning steps before giving a final answer. This simple idea has profound effects: by breaking a problem into smaller steps and letting the model articulate them, it leverages the model's capacity for logical decomposition and reduces mistakes from leaps of thought.

Early demonstrations by Wei et al. (2022) showed that CoT prompting can elicit emergent reasoning capabilities in sufficiently large models. For example, a few exemplar "think-aloud" solutions in the prompt allowed a 540B model to achieve 78% accuracy on GSM8K math problems, whereas without CoT it would be far lower. Crucially, these gains were achieved without any fine-tuning, just clever prompting – indicating that at a certain scale, models have the latent ability to reason if encouraged to do so.

Subsequent work "Zero-shot CoT" even found that simply instructing the model to "Let's think step by step" can trigger CoT reasoning in GPT-3-sized models (although less reliably than using examples).

Beyond linear chains, researchers have developed structured reasoning methods like Tree-of-Thoughts (ToT). Tree-of-Thoughts generalizes chain-of-thought by allowing the model to explore multiple branches of reasoning, backtrack, and evaluate intermediate states. Yao et al. (2023) introduced ToT as a way to do search-based planning with LLMs: the model generates possible "thoughts" at each step, forming a decision tree, then uses a heuristic (which could be the model's own evaluation or a value function) to choose which branch to explore next.

This approach addresses tasks where a single greedy chain might get stuck in a dead-end. The payoff can be dramatic on certain problems – for instance, in the Game of 24 puzzle, GPT-4 with basic chain-of-thought solved only 4% of tasks, whereas Tree-of-Thoughts achieved a 74% success rate by searching through the space of arithmetic operations. Such results illustrate that deliberate planning and lookahead can be integrated into LLM inference, enabling far stronger problem-solving than one-pass generation.

Another refinement of CoT prompting involves the use of special tokens or formatting to clearly delineate reasoning from final answers (the "think" tokens concept). Research on "self-reflection" or tutor-style prompting uses tags like <reason> and <answer> or separate message roles, so that the model knows to keep its reasoning separate from what it tells the end-user.

This can prevent the model from confusing its internal chain-of-thought with the answer (important, for example, in tools or in multi-agent communication where you don't want to expose the raw reasoning to other agents unless intended). It also allows iterative approaches – e.g., the Reflexion framework has the model generate an initial answer and a reasoning trace, then critique its own solution and append feedback to its memory, and finally try again, leading to higher quality answers on tasks like coding puzzles.

In summary, chain-of-thought techniques have become a cornerstone of LLM-as-agent systems. By encouraging stepwise reasoning, they make the agent's decision process more transparent and often more accurate. Empirically, CoT and its variants have yielded state-of-the-art results on many reasoning benchmarks.

For agent developers, CoT prompting is usually the first tool in the toolkit to enhance an LLM's reasoning – it synergizes especially well with tool use and planning (since the model can think about which tool to use next, etc.). However, there are still challenges: CoT traces, if not constrained, can introduce new errors or irrelevant detours (the model might "over-think"). Researchers are thus studying how to better supervise or verify the chains (e.g. using verifiers or critical thinkers that check each step).

Nonetheless, the ability for an LLM to generate "thought tokens" has opened the door to treating it not just as a language mimic, but as a problem-solving entity that can articulate and scrutinize its own reasoning. This ability underpins many of the more advanced agent behaviors, such as tool use and multi-agent dialogue, which rely on the LLM making and communicating complex decisions.

Tool Use and Augmentation of LLMs

While chain-of-thought prompting helps an LLM reason internally, tool use extends its capabilities externally. Even the most powerful LLM has knowledge cut-off dates, limited precision in math, and no direct interface to the world's current information or the ability to take actions. Equipping LLM-based agents with tools has therefore been a major focus, enabling them to retrieve information, execute code, query databases, or interact with the physical world.

Crucially, tool use is often combined with the reasoning patterns above: the agent "decides" when and how to use a tool as a part of its chain-of-thought, then incorporates the tool's output to proceed.

A seminal framework integrating tools is ReAct (Reason+Act) by Yao et al. (2022). ReAct prompts the LLM to produce two interleaved outputs: a reasoning trace and an action. For example, faced with a question like "How many inhabitants does the capital of Australia have?", the agent might think (reasoning): "The capital of Australia is Canberra, I should look that up," and then produce an action: "Search Wikipedia for Canberra population."

The scaffolding executes that search, retrieves the result (e.g. "Canberra… population 456,000…"), and feeds it back into the model, which then continues reasoning with that new info and finally answers. In the ReAct paper, this approach outperformed pure reasoning and pure acting baselines on both knowledge tasks and decision-making tasks.

For question answering tasks like HotpotQA and FEVER fact verification, ReAct was able to retrieve evidence to avoid the hallucinations that a chain-of-thought alone might produce. On interactive tasks (an ALFWorld text-based household simulation and a WebShop e-commerce task), ReAct achieved success rates much higher than imitation learning or reinforcement learning agents, improving absolute success by 34% on ALFWorld and 10% on WebShop over those prior methods.

Another influential work is Toolformer (Schick et al., 2023), from Meta AI, which approached tool-use by fine-tuning the language model itself to insert API calls into its generation. Toolformer provided the model with a set of API schemas (for a calculator, a search engine, a translator, etc.) and had it practice generating text that includes appropriate API call tokens.

The result was an LLM that can "decide for itself which tool to use, when, and how". Without any manual prompting for tools, Toolformer learned to invoke tools when needed, leading to substantially improved zero-shot performance on various tasks compared to a baseline model of the same size. In some cases its performance became competitive with models much larger that lack tool use.

From an industry standpoint, OpenAI's introduction of the function calling API in 2023 made tool integration more accessible. GPT-4 and GPT-3.5 were fine-tuned to output a JSON object when they intend to use a tool, conforming to a predefined function schema. This ensures the output is parseable and reliably triggers the tool.

For instance, if an agent has a lookup_weather(location) function available, the model might output something like: { "action": "lookup_weather", "args": {"location": "San Francisco"} } when asked "Do I need an umbrella tomorrow in SF?". The calling system executes the function and returns the result (e.g. "Rain expected.") which the model can then incorporate into its final answer.

This structured approach has been widely adopted, essentially standardizing the ReAct style (thought→tool→result→...) into an API contract. Microsoft's HuggingGPT can be seen as an extension of this idea to a network of tools: it uses ChatGPT as a controller to pick from hundreds of HuggingFace models (for vision, speech, etc.) to solve multi-modal tasks.

The benefits of tool use in real-world environments are clear. For web-based agents, tools include actions like clicking links, filling forms, or reading page content – essentially treating the web browser as a tool. The WebArena benchmark explicitly provides a set of available web actions (like a web API) that an agent can call to navigate websites and databases.

In coding and software engineering tasks, tools like compilers, linters, and test runners are indispensable. On SWE-Bench, top-performing agents all use tool interactions heavily: for example, they use a Bash tool to run test cases or search the codebase, and an "Edit" tool to modify code files.

Anthropic's Claude-based SWE-Bench agent could execute Python code and shell commands as needed to replicate the developer's workflow (setup, reproduce bug, edit, test). The result was a nearly 50% success rate on the human-verified SWE tasks – a huge leap from <2% success without such agent tooling.

It's worth noting that the integration of tools also brings challenges. The agent must know when to use a tool versus relying on internal knowledge. Using tools can incur latency (e.g. calling an API may be slow), so the agent should not over-use them for trivial matters. There are also failure modes, such as the agent producing an invalid tool call (though function schemas help catch this) or the tool returning unexpected results.

In summary, tool use has proven to be a game-changer for making LLMs into active problem-solvers. Empirical evidence across domains – QA, math, coding, web navigation – shows large gains in capability when LLMs can act on the world. Tools address key weaknesses of LLMs (static knowledge, math errors, inability to execute) by offloading those parts to dedicated systems.

Long-Term Memory and Context Extension

As LLM-based agents operate in more complex scenarios, memory becomes a crucial component. Unlike a human or a classical AI program, a vanilla LLM has no persistence between turns except what you explicitly remind it of in the prompt. This is problematic for long-horizon tasks that may involve hundreds of steps or a vast amount of information that no fixed context window can capture. Thus, researchers have introduced memory systems to allow agents to retain useful information and overcome the limitations of finite context length.

One simple form of memory is the transcript memory – keeping a running log of the conversation or actions and feeding a window of it back to the model each turn. This is essentially what ChatGPT does in a conversation, but it can break down if the history grows too large (older messages get omitted or summarized). Agents often implement summarization or abstraction to compress long histories.

For example, after a lengthy dialogue, the agent might generate a concise summary of important facts so far, and then that summary is used going forward instead of the full log. This ensures important details remain in context. Some systems maintain multiple levels of summary (hierarchical memory), where each new summary itself gets summarized when it grows, etc., creating a layered memory.

However, not everything can be summarized without loss. Vector database memory is a popular solution for semantic recall. In this approach, each chunk of information (e.g. a paragraph the agent read, or a past event) is embedded into a vector and stored in a database. When the agent later needs to remember something relevant, it can embed the query or current context and retrieve the most similar stored chunks (via nearest neighbor search).

Those retrieved items can be inserted into the prompt to refresh the agent's memory. This is often called Retrieval-Augmented Generation (RAG). It has been widely used in open-domain QA and recently in agents. For instance, an agent that has read documentation can store it and later retrieve specific parts when a related question arises, instead of packing the entire documentation into the prompt.

A more AI-driven memory system appears in the Generative Agents work. Each agent in that simulation continuously records every observation and action (forming a very large memory stream). Periodically, the agent uses an importance scoring mechanism to decide which events were significant (e.g. meeting a new person might be marked as important).

Important events are then synthesized into higher-level reflections: for example, "I recall that I have a party to attend tonight, which I'm excited about." These reflections help the agent form plans and influence its behavior (like planning to buy gifts for the party). When the agent faces a situation (e.g. seeing another character), it will retrieve potentially relevant memories (using embedding similarity) such as previous interactions with that character, and that will inform its next action.

In practical benchmarks, memory often shows up as extended context length. Anthropic's Claude and OpenAI's GPT-4 have introduced contexts of 100k or more tokens, which in effect is a short-term memory expansion. With 100k tokens (~75,000 words), an agent can, for instance, load multiple files of a codebase or have a dialogue that lasts several hours without truncation.

Anthropic reported that by using a 200k token context, their SWE-Bench agent let the model itself decide how to navigate and modify a large code repository, stopping only when it believed it had solved the issue (or ran out of context). This is a form of "brute force" memory – just give the model all the data at once. It clearly helps (the model can directly refer back to earlier parts of the conversation or code), but it's expensive and still bounded.

Memory is also tied to planning and reflection mechanisms. The Reflexion approach by Shinn et al. had the agent store a sort of memory of what went wrong in previous trials. For example, if the agent attempts to solve a puzzle and fails, it writes to its memory: "I tried X and it didn't work because Y." On the next attempt, it reads that and avoids the same mistake, or tries a new angle. This is episodic memory – remembering outcomes of past attempts. It moves the agent closer to how a person might learn from experience (without updating weights, just by remembering in plain text).

Despite these advances, memory in current LLM agents is still rudimentary compared to human memory. Agents often lack a true long-term memory that persists across sessions (each run might start fresh except for what the developers preloaded). There are also issues of memory accuracy and consistency – an agent might "remember" something incorrectly or retrieve irrelevant info.

If the memory grows too large, searching it becomes slow or yields too many candidates to include in context. Moreover, if not carefully managed, an agent with a lot of memory may start to confuse which information came from where (was it told by the user, did it read it, or was it an assumption?). Ensuring that the agent trusts the right sources and not stale or false memories is an open problem.

In summary, memory modules are essential for long-horizon and complex tasks, allowing agents to accumulate knowledge over time and not reset each interaction. Techniques like summarization, vector database retrieval, and reflection buffers extend the effective context beyond the LLM's built-in window. As research continues, we expect more sophisticated memory architectures to emerge, because enabling agents to learn and adapt over time is a key step toward autonomy.

Architectures for Agents and Orchestration of Sub-Agents

Building a full autonomous agent typically involves orchestrating multiple components – possibly multiple LLMs or a mix of LLMs and non-LLM modules – into an agent architecture. We now survey how researchers have structured these architectures, especially focusing on systems with sub-agents or hierarchical organization, and how they coordinate complex workflows.

One straightforward architecture is a planner-executor split. In this design, one part of the system (which could be an LLM prompt or a separate LLM) takes the high-level task and breaks it into a plan or sequence of steps. Another part then carries out each step. This was seen in some early OpenAI experiments: e.g., using GPT-4 to first plan ("First, I will search for X, then I will …") and then follow that plan step by step via function calls.

Microsoft's Jarvis (HuggingGPT) explicitly had Stage 1: Task Planning and Stage 3: Task Execution separated. The benefit of this separation is that the planning phase can consider the whole task and perhaps see the end goal, while the execution phase can focus on one subtask at a time. If the execution runs into trouble, it can even loop back and ask the planner for adjustments.

Another architecture pattern is the controller-worker model. Here, a single "main" agent (LLM) serves as a controller that can call upon various worker modules. The workers might be other LLMs specialized for certain domains (e.g. a code generation model, a math model) or even non-LLM tools as described earlier.

HuggingGPT is a prime example: ChatGPT was the controller and the various HuggingFace models (for vision, speech, etc.) acted as workers, with ChatGPT deciding which worker to invoke for each subtask. Similarly, say in a robot context, an LLM might be the brain that decides to use one module for navigation and another for grasping. This controller-worker setup centralizes decision-making, which can simplify coordination but also puts a lot of responsibility on the controller's prompt or training.

A more decentralized approach is a multi-agent collaborative architecture, which we discuss more in the next section. But it's worth noting here that even within one "agent", designers sometimes include multiple LLMs that play complementary roles. For example, one LLM could be a Generator (proposes candidate solutions) and another the Evaluator/Critic (judges the solutions and asks for revisions).

This has been explored in the context of code generation and essay writing – the "two LLMs in a loop" idea, where one generates output and the other provides feedback until the output is good. Such setups leverage the idea that LLMs can help correct each other's errors or refine ideas (one might catch a flaw the other missed).

An interesting recent trend is encoding human workflows or methodologies into the agent prompts as a form of architecture. The MetaGPT framework introduced the idea of using Standard Operating Procedures (SOPs) and role assignments to structure the interaction of multiple GPT-based agents.

Essentially, MetaGPT acts like a virtual software company: when given a user's request for a new software, it assigns an LLM to be the PM (who gathers requirements), one to be the Architect (who drafts a design), multiple to be Engineers (who write code for different components), etc. They pass documents (prompts) to each other in an order defined by an SOP (like a design doc goes to engineers, code goes to a reviewer, etc.).

This meta-programming approach imposes a workflow graph on agent interactions rather than free-form chatting. By doing so, MetaGPT reported generating more coherent and correct solutions on collaborative programming tasks than simpler multi-agent chats.

From an engineering standpoint, the "scaffolding" around the LLM is a crucial part of the architecture. This includes prompt templates, loops for tool use, error handling, etc. As noted by Anthropic in their SWE-Bench agent description, the scaffolding can greatly affect performance.

Two teams might use the same base LLM (e.g. Claude) but get very different results on a benchmark depending on how well they design the agent's loop and prompts. For example, some scaffolds might hard-code a particular sequence: "always do step1 then step2 then step3," whereas a more flexible scaffold lets the model decide the next action freely (which could be better or worse depending on the model).

To summarize agent architectures: hierarchy and modularity are key themes. Whether it's a simple two-tier planner/executor, a hub-and-spoke controller with tools, or a complex multi-agent workflow, breaking the problem down and assigning pieces either to different agents or different phases leads to better performance and manageability.

Empirically, many tasks that a single monolithic LLM fail at can be solved by a composed system of LLM+tools or multiple specialized LLMs. For example, in the ReAct paper the interactive decision tasks (WebShop, ALFWorld) were nearly unsolvable by a single-step model, but a reasoning+acting loop solved >50% of them.

Multi-Agent Collaboration and Role-Playing

Thus far we often spoke about an "agent" in singular, but a very intriguing frontier is the interaction of multiple LLM agents with each other. Multi-agent systems can unlock scenarios that a lone agent cannot handle, either because tasks can be done in parallel, or because agents can critique and improve each other, or because the task itself inherently involves multi-party interaction (e.g. negotiation, dialogue, games).

One line of work uses multi-agent dialogues as a means to boost performance on single-agent tasks. For instance, the CAMEL framework (Li et al. 2023) sets up two ChatGPT agents, one as a "user" and one as an "assistant," and gives them a task to solve together with minimal human input.

The user-agent is programmed (through an "inception prompt") to always drive the conversation towards the goal, and the assistant-agent tries to comply and provide information. They effectively brainstorm and exchange ideas. This role-playing technique turned out to generate high-quality conversational data that can be used to fine-tune models or to directly solve problems that normally require human in the loop.

Notably, CAMEL showcased that autonomous cooperation is possible: the agents remained largely on track with human intentions and were able to follow instructions collaboratively to reach a solution. This is encouraging because it hints that multi-agent systems might not always devolve into incoherence; with the right roles and prompts, they can be harnessed systematically.

Multi-agent communication also shines in environments where agents represent different entities. A striking example is Meta's CICERO, which is an agent that achieved human-level performance in the game Diplomacy by negotiating with multiple human players.

Diplomacy is a seven-player board game where natural language negotiation is central – players form alliances, share plans, and bluff. CICERO integrated an LLM for dialogue with a strategic planning module that modeled the game state and other players' likely actions.

Essentially, it had a pipeline: predict others' moves, plan an optimal joint strategy (which may involve proposing certain alliances), then generate appropriate messages to communicate that to each player. Over 40 online games, CICERO ranked in the top 10% and more than doubled the average score of human players, a performance that garnered a Science publication.

This required delicate coordination: the language model had to produce persuasive, coherent dialogue consistent with the plans, and it had to maintain trust with humans over dozens of exchanges. The success of CICERO demonstrates that LLM-driven agents can coordinate with others (including humans) in complex, competitive-cooperative settings. It's a proof-of-concept that multi-agent communication isn't just for toy problems – it can handle something as messy as human negotiation, given the right scaffolding of planning and RL to guide the language model.

Another fascinating study is Generative Agents by Park et al., which we mentioned earlier for its memory component. In that work, 25 LLM agents inhabited a simulated town (like The Sims) and could interact via dialogue. They formed social connections, shared information, and even organized a group activity (a Valentine's Day party) purely through multi-agent local interactions.

One agent started with a notion to throw a party; it told a few others, who then told others, and so on, invitations spread organically. Agents asked each other out on dates to the party and coordinated to attend at the right time, all without a central script. The researchers found that these emergent social behaviors were markedly more human-like when the architecture included proper observation, planning, and reflection for each agent.

Multi-agent collaboration can also be used to boost factual correctness and reduce errors. One method is to have agents double-check each other's outputs. For example, in a question-answering scenario, you could spawn multiple agents, have each propose an answer with reasoning, and then have them discuss or vote on the final answer.

This idea of an "AI debate" or "committee of agents" has been floated in alignment research as well – the hope is that if one agent is about to hallucinate, another might catch it. There has been work like "Self-Critique and Revision" where an agent generates a solution and another agent (or the same agent in a different mode) critiques it, then it revises, and so forth, leading to an improved answer that is then final.

Coordination mechanisms in multi-agent systems often revolve around communication protocols. The simplest is turn-taking conversation: agents speak to each other in natural language in a round-robin or as needed. This is flexible but can be inefficient if not managed (agents might chat in circles).

Some frameworks implement a facilitator that filters or directs communications – e.g. a special agent that monitors if the others are converging or if the conversation needs a nudge. In more formal approaches, one might use something like shared memory: all agents write their current state or intention to a common board, and read what others have written. This ensures everyone has the same knowledge.

Role assignment is crucial to avoid every agent doing the same thing or arguing constantly. By giving them complementary roles or information, you encourage cooperation. For instance, one agent could have access to certain tools or knowledge that another doesn't, so they need each other.

Despite these challenges, the trajectory is promising. Multi-agent approaches have produced novel capabilities: from negotiation (CICERO), to social simulation (Generative Agents), to improved task performance via cooperation (CAMEL role-playing). They also open up new possibilities like using one agent to align or supervise another (for safety, one could imagine an agent that monitors another's outputs for policy compliance – in fact, Anthropic's "Constitutional AI" can be seen as an AI judge providing feedback to the main AI).

We are essentially witnessing the early days of LLM agent societies, where the focus is shifting from a single powerful genius to a team of AI workers each with limited skills but good communication. This mirrors how human organizations solve problems and could be key to scaling AI capabilities further.

Benchmarks and Evaluation Environments

To measure progress in this fast-evolving field, new benchmarks and evaluation suites have been developed that test LLM-based agents in long-horizon, realistic scenarios. Traditional NLP benchmarks (like static QA pairs or single-turn tasks) do not capture the interactive and process-oriented nature of agent behavior. Here we highlight some influential benchmarks that researchers use to assess reasoning, tool-use, and multi-agent coordination capabilities of LLM agents.

SWE-Bench (Software Engineering Bench)

SWE-Bench focuses on real-world software engineering tasks with 2,294 issues from GitHub repositories. The agent's job is to generate code edits to resolve bugs or implement feature requests. This requires understanding how multiple files interact, making changes, running tests, and handling errors. Evaluation is via unit tests—the agent passes if all tests run successfully after its changes.

When first released, even top models struggled: Claude 2 solved only 1.96% of issues. Later, a verified subset of 500 more solvable issues was released, where progress has been faster: Claude 3.5 achieved 49% success, and a combined approach with Claude 3.7 and OpenAI models reached 65.4% by early 2025.

WebArena

WebArena simulates the open web with various websites: e-commerce, forums, wikis, maps, etc. Agents perceive web pages and take human-like actions (clicking, typing, scrolling). Tasks are given as natural language instructions requiring multi-step web interaction.

For example: "Find art museums in Pittsburgh, create an itinerary with minimal driving distance, then log the plan in a repository." This tests planning, tool use, and state tracking across multiple websites. The benchmark provides programmatic verification to check if tasks are completed correctly.

AgentBench

AgentBench evaluates LLMs across 8 different environments, spanning operating systems, databases, knowledge graphs, games, puzzles, household tasks, and web browsing. This comprehensive benchmark covers code-grounded, knowledge-grounded, and web-grounded tasks.

Testing 29 different LLMs revealed a substantial gap between proprietary and open-source models. GPT-4 achieved the highest score (4.01), while the best open model (Llama2-70B-chat) scored only 0.78. This confirmed that today's strongest agents are powered by frontier models, with smaller models struggling on complex multi-turn tasks.

Other Evaluations

Other notable benchmarks include TauBench (testing real-world reliability), LEval (for long-form tasks), and specialized evaluations for robotics and embodied agents. These benchmarks increasingly focus on the process rather than just the final output—logging action traces to understand where and why agents fail.

A key challenge is reproducibility in interactive environments. Benchmarks address this with standardized simulation code, fixed random seeds, and clear success criteria. This standardization ensures fair comparison across different research teams and approaches.

The advent of these benchmarks marks a turning point, providing testbeds for autonomous agents that closely resemble real-world applications. While current performance is far from perfect, these benchmarks drive progress by enabling rigorous evaluation of reasoning, tool use, memory, and collaboration capabilities.

Current Progress and Synthesis of Results

Bringing together the threads from research and benchmarks, we can map out the current progress in combining LLM reasoning with structured scaffolding (tools, orchestration, etc.) to create capable agents. Over roughly the past two years, there have been rapid improvements, with notable milestones along the way:

Emergence of Reasoning in LLMs (2022)

The discovery of chain-of-thought prompting unlocked complex reasoning in models like GPT-3 and PaLM. This set the stage by showing large LMs can handle multi-step logic and math if prompted well. Around the same time, techniques like self-consistency (where multiple CoT answers are sampled and the most common is chosen) improved reliability.

By late 2022, the idea of using LLMs as decision-makers had taken hold, illustrated by ReAct's success in combining thought and action. Models were still often brittle, but proof-of-concept was there: e.g. in ALFWorld (a text game), a GPT-3 with ReAct prompting solved many levels, outperforming a traditional RL agent that had been trained on those levels for many steps.

Tool Use and APIs (2023)

With the release of ChatGPT and GPT-4, and the introduction of function calling, tool use became mainstream. ChatGPT Plugins (Web browsing, Code Interpreter) demonstrated impressive versatility – a GPT-4 with a code execution tool could solve data analysis tasks end-to-end, writing Python code, running it, and analyzing results.

On open-source, projects like LangChain made it easier to prototype agents that use tools and memory. By mid-2023, numerous "AutoGPT"-style agents were popping up, leveraging GPT-4's capabilities to multi-step execute user goals (web search being a common use-case).

However, these early fully-autonomous attempts also revealed the limitations: AutoGPT, for instance, often got stuck in loops and struggled with tasks requiring precise understanding. Users reported it would repeat steps or generate irrelevant actions, highlighting that simply wrapping an LLM in a loop doesn't guarantee success without careful prompt engineering and safeguards.

Academic Research on Agents (2023-2024)

Top conferences saw a surge of papers on LLM-based agents. For example, HuggingGPT illustrated a way to solve multi-modal tasks by composing existing models via an LLM. Tree-of-Thoughts and Reflexion improved the intrinsic reasoning and self-correction abilities of single agents.

Voyager (Wang et al. 2023) showcased an agent that uses GPT-4 to autonomously explore Minecraft, writing and executing code to accomplish open-ended goals, and it managed to discover dozens of tools in the game and remember them – essentially an embodied lifelong learning agent.

In multi-agent, CAMEL validated cooperative role-play as a data generation and problem-solving tool, while MetaGPT provided perhaps the most rigorous systemization of multi-agent collaboration by mimicking human organizational structures.

We also saw work addressing safety and reliability in agents: for instance, some benchmarks tested if an agent could perform dangerous actions given the opportunity (as an alignment evaluation). This led to mitigations like limiting tools or inserting guardrail prompts.

State-of-the-Art Capabilities (early 2025)

Today, the best LLM agents (e.g. those built on GPT-4 or Claude 2/3) can achieve moderate success on complex benchmarks when well-orchestrated. To give a snapshot: on AgentBench, GPT-4 solved many tasks across 8 domains, vastly outperforming any open model (closing the gap between GPT-4 and humans on those tasks is the next challenge).

On SWE-Bench, agents built around Claude have moved from solving ~2% to ~50% of the verified issues in one year, a remarkable improvement, though still a long way from human-level (for reference, a competent human developer would likely solve >90% given the same information).

On Web tasks (WebShop/WebArena), GPT-4 based agents currently lead – one analysis of web agent traces noted that even they are far from perfect, often failing on fine-grained page understanding. Multi-agent simulations like Generative Agents remain mostly in the research/demo stage (not benchmarked in a competitive way), but they did show plausibility of long-running social interactions.

An interesting emergent theme is that improving agents is not just about improving base LLMs, but equally about better orchestration and training. As AgentBench authors noted, even some relatively small finetuned models performed decently on certain tasks by virtue of being trained on dialogues or instructions that simulate multi-turn interactions.

And scaffolding tricks – such as better prompt wording, using a scratchpad to let the model track subtasks, etc. – can yield significant gains without any model change. This means part of the progress comes from the creative use of LLMs rather than raw model breakthroughs.

In summary, the current state is that LLM-based agents can achieve non-trivial performance on complex, long-horizon tasks, especially when equipped with tools and structured reasoning, but they are not yet consistent or reliable enough for unrestricted real-world autonomy. Each impressive demo often comes with caveats: maybe it works 8 out of 10 times, or it requires human oversight to correct occasional mistakes.

Nonetheless, compared to a few years ago when even solving a multi-step reasoning puzzle was hit-or-miss, we now have agents completing moderately complicated objectives like "book a trip given some constraints" or "fix this bug in the code" or "coordinate with another agent to trade information". The combination of CoT (thinking), tools (acting), memory (remembering), and multi-agent collaboration (cooperating) has pushed the envelope of what AI systems can do on their own.

Multi-Agent Coordination and Social Dynamics

When multiple agents are involved, a host of new challenges appear. One is coordination without collision – ensuring agents don't all try to do the same subtask or interfere with each other. In human teams, this is managed by communication and roles, as we do with AI roles too, but still an agent might decide to go rogue or misinterpret another agent's message.

There's a risk of feedback loops where agents amplify each other's mistakes (as mentioned, one lies and others propagate it). There's also the issue of inefficiency: two agents might spend a lot of time chatting and not enough doing actual work. Setting proper objectives for each agent is vital so that the conversation converges.

In CAMEL's work, they needed "inception prompting" to bootstrap the conversation in a productive way; without that, the agents might have fallen into banal talk or misunderstanding. For more than two agents, this gets even harder – managing a multi-agent discussion is like herding cats. Maybe one agent (a coordinator) is needed to keep meeting minutes and summarize consensus, much like a moderator in group chats.

Another challenge is trust and truthfulness in multi-agent systems. If agents are not aligned (for example, some are set as adversaries or competitors, like in Diplomacy or debate settings), we have to ensure that deceptive strategies don't break the evaluation or the intended outcome.

Cicero had to balance deception and honesty carefully to play Diplomacy well without ruining long-term cooperation. That required a model of the other players' beliefs – something current LLMs don't inherently have (they're just predicting text without a concept of another agent's mind). We might need to imbue agents with a theory-of-mind module to reason about what others know or intend. Some initial work in this direction has started, but it's early.

Despite these challenges, the potential of multi-agent systems is vast. By distributing cognitive tasks among specialized agents, we can tackle problems too complex for a single agent. By having agents check each other's work, we can reduce errors and improve the overall quality of solutions.

And by modeling social dynamics between agents, we can create simulations that help us understand human social behavior or design systems that better interact with humans in group settings. As LLMs continue to advance, these multi-agent frameworks will likely become increasingly sophisticated, leading to AI systems that can truly collaborate in human-like ways.