AI Agent Orchestration: How Multiple AI Agents Are Coordinated to Solve Problems

A single question to an AI chatbot usually gets a single answer. A growing share of AI software works differently. One request can set off a chain of model calls. A planning step splits the job into pieces. Separate "workers" run web searches, read documents, write code or check one another's output, and a final step puts the results back together. Getting several AI components to cooperate like this is called AI agent orchestration.

The term is new, but the underlying problem is old. Computer scientists have studied how independent software "agents" can share work since at least the late 1970s. Today's large language models (LLMs) have given that research a second life. This article explains what agents and orchestration are and where the core ideas came from. It covers the main coordination patterns in use today, the protocols that connect agents to tools and to each other, and the costs, failure modes and safety questions that come with the approach.

What is an AI agent?

The word "agent" gets used loosely, so it helps to begin with definitions.

In classical computer science, a multi-agent system (MAS) is, in Wikipedia's words, "a computational system composed of multiple interacting intelligent agents." Such systems can solve problems that would be difficult or impossible for one agent or one monolithic program. The agents can be software, robots or even humans. The Wikipedia article on multi-agent systems lists three defining traits:

• Autonomy. Each agent is at least partly independent.
• Local views. No agent has a complete, global view of the whole system. Often the system is too complex for any single view to be useful.
• Decentralization. No single agent controls everything. If one did, the article notes, the system would in effect become monolithic again.

In the current LLM era, the AI company Anthropic offered a practical definition in its December 19, 2024 engineering post "Building effective agents." It separates two kinds of system:

• Workflows, in which "LLMs and tools are orchestrated through predefined code paths." A programmer decides the order of steps in advance.
• Agents, in which LLMs "dynamically direct their own processes and tool usage," keeping control over how they complete a task.

This line matters for orchestration. Some "agent orchestration" is really workflow design: an engineer wires model calls together in a fixed order. Other systems let a model decide, while running, which subtasks to create and which tools to call. Most real products mix the two.

A short history: from blackboards to contract nets

The ideas behind modern orchestration were worked out long before chatbots.

The blackboard architecture

One of the earliest coordination designs is the blackboard system. The Wikipedia article describes it as an approach in which a shared knowledge base, the "blackboard," is updated again and again by a group of specialist knowledge sources until a solution emerges. It was designed for complex, poorly defined problems. A blackboard system has three parts:

1. Knowledge sources, independent specialist modules that each contribute one kind of expertise.
2. The blackboard, a shared store of the problem, partial solutions, suggestions and other contributed information.
3. A control shell, which manages the flow of problem-solving so that the specialists act in a sensible order.

The usual comparison is a group of experts standing around a chalkboard. Each one steps up when they see something they can add, and their additions may prompt someone else to act. The speech-understanding system Hearsay-II is among the best-known early academic blackboard systems and was described in a 1980 ACM Computing Surveys article. In 1985 Barbara Hayes-Roth published the BB1 architecture, which applied the same opportunistic approach to the system's own control, so it could notice when planning stalled and switch strategies. Blackboard designs were later used in the planning component of the RADARSAT-1 satellite mission control system, in Adobe Acrobat Capture's page recognition, and in military command-and-control systems that detect and track objects. They are still a standard tool in video game AI.

The blackboard pattern lives on in LLM systems wherever several agents read from and write to a shared document, database or "scratchpad" instead of messaging each other directly.

The Contract Net Protocol

The second classic idea is about dividing up work. In 1980 Reid G. Smith introduced the Contract Net Protocol (CNP), a task-sharing protocol for multi-agent systems. Each agent plays one of two roles, manager or contractor, and the protocol works much like a sealed-bid auction:

1. The manager sends out a call for proposals describing a task.
2. Contractors that are interested reply with a proposal that contains the information the manager needs to decide. Uninterested contractors send a refusal.
3. The manager picks the best proposal, accepts it and rejects the others.
4. The winning contractor does the task and sends back an inform message with the results, or a cancel message if it cannot finish.

The Foundation for Intelligent Physical Agents (FIPA), a standards consortium, implemented the protocol in its Agent Communication Language and also published an "iterated" version that allows several bidding rounds. The manager/contractor split appears again, almost unchanged, in today's "orchestrator-worker" LLM designs.

Agent communication languages

If agents are to cooperate, they need a shared way to talk. In the 1990s researchers built agent communication languages based on speech act theory, the philosophical idea that utterances are actions such as requesting, informing or promising. Two became well known:

• KQML (Knowledge Query and Manipulation Language), described in a 1994 paper at the CIKM conference.
• FIPA-ACL, from the Foundation for Intelligent Physical Agents. It defines a set of performatives, or communicative acts, each with a stated meaning.

Both need the agents to share an ontology, a common definition of the things they can talk about and how those things relate. The Java Agent Development Framework (JADE) implemented the FIPA standards. Wikipedia notes that neither FIPA nor the related OMG MASIF standard is actively maintained today. That gap helps explain why new protocols appeared in 2024 and 2025, as described below.

Why orchestrate multiple agents at all?

If one powerful model can answer a question, why split the work up? The research and industry sources point to several reasons.

Context limits. An LLM can only consider a fixed amount of text at once, its context window. A long task can overflow it. Anthropic's June 13, 2025 write-up of its multi-agent Research feature explains that each subagent works in its own context window. Several subagents can therefore explore different parts of a question without crowding one another out, and the lead agent receives only their condensed findings.

Parallelism. Independent subtasks can run at the same time. Anthropic reported that running subagents in parallel, and letting them make tool calls in parallel, cut research time by up to 90% on complex queries.

Specialization. Different steps can get different instructions, tools or even different models. Anthropic's routing pattern, for example, sends easy questions to smaller, cheaper models and harder ones to more capable models.

Checking and quality control. One agent can review another's work. A separate reviewer catches errors that the author misses.

Breadth. Some questions are naturally "breadth-first": they require looking at many sources or options. Anthropic found that its multi-agent system outperformed a single Claude Opus 4 agent by 90.2% on its internal research evaluation, with the biggest gains on such breadth-first queries. This is a company's own internal benchmark, not an independent result, but it is one of the clearer published measurements of the trade-off.

Anthropic's analysis of the BrowseComp benchmark adds a revealing detail. The amount of tokens (units of text) a system used explained about 80% of the difference in performance. Much of the benefit of multiple agents seems to come from letting the system spend more total effort on a problem than a single context window allows.

Core orchestration patterns

Anthropic's "Building effective agents" post sets out a vocabulary of patterns that has become widely referenced. They run roughly from most to least predictable.

Prompt chaining

A task is split into a fixed sequence of LLM calls, and each call works on the previous call's output. Programmatic checks, which Anthropic calls "gates," can be placed between steps to confirm that things are on track. A typical example is drafting an outline, checking that it meets set criteria, and then writing the full document from the outline. The trade-off is clear: more steps mean more latency, but each step is easier for the model to get right.

Routing

An input is first classified and then sent to a specialized follow-up process. A customer-service system might send refund requests, technical questions and general inquiries down different paths, each with its own prompt and tools. Routing is also a cost tool, since simple requests can go to cheaper models.

Parallelization

Several LLM calls run at once and their outputs are combined. Anthropic describes two versions:

• Sectioning, where a task is split into independent parts that run side by side. For example, one call screens a request for inappropriate content while another writes the response.
• Voting, where the same task runs several times to get different outputs. For example, several reviewers each check code for security vulnerabilities, and a problem flagged by any of them gets attention.

Orchestrator-workers

A central LLM breaks a task down while it runs, hands the pieces to worker LLMs and then combines their results. The key difference from parallelization is that the subtasks are not fixed in advance; the orchestrator decides them based on the input. Anthropic's example is a coding change that touches many files, where the number and nature of the edits depend on the request. This is the modern descendant of the Contract Net manager, although the orchestrator usually assigns work directly instead of running an auction.

Evaluator-optimizer

One LLM produces a response and another evaluates it and gives feedback, in a loop. It works best when there are clear evaluation criteria and when repeated refinement measurably improves the result, much as a human writer improves a draft after an editor's notes.

Autonomous agents

At the open end of the range is the fully autonomous agent. Once the task is clear, it plans and acts on its own, checks the results from its environment at each step, and pauses for human input at checkpoints or when it gets stuck. Anthropic says such agents suit open-ended problems where the number of steps cannot be predicted. It also warns that they bring higher costs and a risk of compounding errors, and it recommends extensive testing in sandboxed environments along with guardrails.

The building block: reasoning plus acting

Most LLM agents rest on a simple loop: think about what to do, take an action such as a search or a calculation, look at the result, and repeat. An influential version of this loop is ReAct, short for "Reasoning and Acting." It was introduced in the paper "ReAct: Synergizing Reasoning and Acting in Language Models" by Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan and Yuan Cao, first posted on October 6, 2022 and published at the ICLR conference.

ReAct has the model produce reasoning traces and task actions in alternation. The reasoning helps it plan, keep track of progress and recover from surprises. The actions let it pull in outside information. In question-answering and fact-checking tests, giving the model access to a simple Wikipedia API reduced the hallucination and error propagation seen in pure "chain-of-thought" reasoning. In two interactive decision-making benchmarks, ALFWorld and WebShop, ReAct beat imitation-learning and reinforcement-learning methods by an absolute success rate of 34% and 10% respectively, using only one or two worked examples in the prompt.

An orchestrated system is, in a sense, many of these loops arranged together, with a coordinator deciding which loop runs when and how their outputs combine.

Landmark systems and frameworks

AutoGPT (2023)

The public's first big exposure to autonomous agents came with AutoGPT, released on March 30, 2023 by Toran Bruce Richards, founder of Significant Gravitas Ltd. It was an open-source program built on OpenAI models such as GPT-4. The user gave it a name, a role, a main goal and up to five sub-goals. AutoGPT then broke the goal into subtasks and worked through them with little further input, browsing the web, managing files and keeping a short-term memory of the current task.

AutoGPT also made the problems of autonomy visible. According to Wikipedia, it could get stuck in loops repeating the same subtask, which developers blamed on its poor memory of what it had already done. It could hallucinate, presenting false information as fact. Because it relied on its own feedback, mistakes could pile up, and users could not easily correct it partway through. And because every step called a paid API, costs added up. Andrej Karpathy linked its performance problems to its finite context window, saying it can "go off the rails." These weaknesses shaped later designs, which put more weight on structure, checkpoints and human oversight.

AutoGen (2023)

Also in 2023, a team of researchers including Qingyun Wu, Gagan Bansal, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger and Chi Wang posted "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation" (arXiv 2308.08155, first submitted August 16, 2023). AutoGen is an open-source framework in which developers build applications from several "customizable, conversable" agents that talk to one another to complete a task. Agents can combine LLMs, human input and tools, and developers can define the conversation patterns using both natural language and code. The paper reports example applications in mathematics, coding, question answering, operations research, online decision-making and entertainment.

AutoGen's main idea is that multi-agent coordination can be treated as a conversation. A "coder" agent and a "critic" agent, for example, can exchange messages until the code passes review.

CAMEL and the research wave

Wikipedia's multi-agent systems article also points to CAMEL (2023), an early framework for LLM-based "communicative agents" that role-play to cooperate on tasks. It also cites MALLM (2025), a framework for systematically evaluating different LLM multi-agent setups. The fact that evaluation frameworks exist shows the field moving from demos toward measurement.

A production example: Anthropic's Research feature

One of the more detailed public descriptions of a production multi-agent system is Anthropic's June 2025 engineering post about its multi-agent research system. It works as follows:

1. A lead agent (Claude Opus 4) reads the user's question, makes a plan and saves it to memory.
2. It creates several subagents (Claude Sonnet 4), each searching a different part of the question in parallel with its own context window.
3. The lead agent combines their findings and decides whether more research is needed.
4. A separate CitationAgent goes through the final report and links its claims to sources.

The team's lessons are useful for anyone building such a system. Subagents needed detailed task descriptions: an objective, an output format, guidance on tools, and clear boundaries. Without them, agents duplicated each other's work or left gaps. The team wrote "effort-scaling" rules into the prompts so that a simple question got one agent and a few tool calls, while a complex one got more subagents. Early versions made classic mistakes. They spawned too many subagents for simple questions, searched endlessly for sources that did not exist, and kept searching after they already had enough. The team also found that small changes to the lead agent could change subagent behavior in ways that were hard to predict.

Connecting agents to tools and to each other: new protocols

Because the older FIPA-era standards are no longer maintained, the LLM industry has started building new plumbing.

Model Context Protocol (MCP)

On November 25, 2024, Anthropic announced the Model Context Protocol (MCP), an open-source standard for connecting AI assistants to the systems where data lives, such as content repositories, business tools and development environments. Its goal is to replace one-off custom integrations with a single protocol. MCP has two roles:

• MCP servers, which expose data or capabilities.
• MCP clients, the AI applications that connect to them.

At launch, Anthropic said Block and Apollo had integrated MCP, and that developer-tool companies Zed, Replit, Codeium and Sourcegraph were building it into their platforms. The company also released open-source servers for services including Google Drive, Slack, GitHub, Git, Postgres and Puppeteer. In orchestration terms, MCP standardizes the link between an agent and its tools.

Agent2Agent (A2A)

On April 9, 2025, Google announced the Agent2Agent (A2A) protocol, an open protocol that lets AI agents communicate, exchange information securely and coordinate actions across platforms, whatever vendor or framework built them. Google said more than 50 technology partners contributed, including Atlassian, Box, Salesforce, SAP and Workday, along with service firms such as Accenture, Deloitte and KPMG. Its main features are:

• Built on existing standards: HTTP, Server-Sent Events (SSE) and JSON-RPC.
• Agent Cards: JSON documents in which an agent describes its capabilities, so that a client agent can find the right agent for a job. This is a modern version of the "capability advertisement" that contractors make in a contract net.
• Tasks with a lifecycle: a task may finish immediately or run for hours or days, with agents keeping each other updated on its status. A task's output is called an "artifact."
• Support for long-running work, enterprise authentication, and more than just text, including audio and video streaming.

Google describes A2A as complementary to MCP: MCP gives an agent tools and context, while A2A handles communication and collaboration between agents. Together they fill roughly the gap that KQML and FIPA-ACL tried to fill in the 1990s.

Costs and trade-offs

Orchestration is not free, and the clearest numbers come from the people building these systems.

Anthropic reported that agents typically use about 4 times as many tokens as ordinary chat interactions, and multi-agent systems about 15 times as many. Since providers charge by the token, it concluded that multi-agent systems make economic sense only for tasks valuable enough to justify the cost. Some tasks also do not split well. Work where every step depends closely on the others, or where all agents need the same context, gains little from being divided up.

That is why Anthropic's design advice starts with restraint: use the simplest solution that works, and add complexity only when it clearly improves results. It suggests developers begin by calling LLM APIs directly, since many patterns need only a few lines of code. It cautions that agent frameworks can hide the actual prompts and responses, which makes debugging harder.

Common failure modes

The literature and the post-mortems point to several recurring problems.

• Loops and runaway effort. AutoGPT could repeat the same subtask. Anthropic's early research agents kept searching after they had enough information.
• Compounding errors. In a long chain of steps a small early mistake can grow. Wikipedia notes this for AutoGPT, and Anthropic says errors compound in long-running, stateful processes. Its fix was to add checkpoints, the ability to resume, and retries.
• Poor task division. Vague instructions to subagents led to duplicated effort and coverage gaps.
• Bad tool descriptions. Anthropic found that poorly described tools sent agents down the wrong paths. It advises giving tool definitions as much care as the main prompts, and "poka-yoke" design, a Japanese manufacturing term for mistake-proofing. In one coding project, requiring absolute file paths instead of relative ones eliminated a whole class of errors.
• Hard debugging. Agent runs are not deterministic, so the same input can play out differently. Anthropic relied on full production tracing of agent decisions to find problems.
• Bottlenecks. When the lead agent has to wait for every subagent to finish, the slowest worker sets the pace. Anthropic said it planned to address this with asynchronous execution.
• Source quality. Human testers noticed agents favoring search-optimized content farms over authoritative sources, which automated checks had missed.

Evaluating orchestrated systems

Testing a multi-agent system is harder than testing a single prompt, because many different sequences of steps can lead to a good answer. Anthropic's advice:

• Start small, with around 20 representative test queries, instead of waiting to build a large test suite. Early changes often have large effects that a small set will reveal.
• Use an LLM as a judge with a rubric covering factual accuracy, citation accuracy, completeness, source quality and tool efficiency. A single judging prompt that gave a score from 0.0 to 1.0 plus a pass/fail grade matched human judgment best.
• Keep humans in the testing loop to catch edge cases that automation misses.
• For agents that change things in the world, judge the end state (did the right thing happen?) rather than whether the agent followed a particular sequence of steps.

Governance, safety and human oversight

As agents gain the power to act by sending messages, editing files or moving money, orchestration becomes a governance question as well as an engineering one.

A widely used reference point is the U.S. National Institute of Standards and Technology's AI Risk Management Framework (AI RMF 1.0), released on January 26, 2023. It is organized around four functions: Govern, Map, Measure and Manage. On July 26, 2024, NIST added a Generative AI Profile (NIST-AI-600-1) to apply the framework to generative systems. The framework is voluntary and not written specifically for agents, but its structure carries over well. Organizations need to know which agents exist and what they can touch (map), test how they behave (measure), set controls and responses (manage), and assign accountability (govern).

The practical guidance from builders points in the same direction. Anthropic recommends that autonomous agents pause for human feedback at checkpoints or when blocked. It also recommends testing them in sandboxes before they get real-world access, and keeping their planning steps visible. Transparency, meaning the ability to see what each agent did and why, appears both in Anthropic's design principles and in its production use of tracing.

The underlying tension is the one the old multi-agent literature identified. Decentralization and autonomy make a system flexible and powerful. They also make it harder for any single party, human or machine, to see and control everything. Good orchestration keeps the benefits of division of labor while keeping a clear line of sight to what the system is doing.

Frequently asked questions

Is "agent orchestration" the same as using several chatbots at once? Not quite. Orchestration means the components are coordinated: something decides which agent does what, in what order, and how the results combine. That "something" can be fixed code (a workflow) or another LLM acting as orchestrator.

Does using more agents always give better results? No. Anthropic reported large gains on breadth-first research questions, but also about 15 times the token use of a chat. It advises using the simplest approach that works. Tasks with tightly linked steps may not benefit at all.

What is the difference between MCP and A2A? MCP, announced by Anthropic in November 2024, connects an AI application to tools and data sources. A2A, announced by Google in April 2025, lets agents built by different vendors communicate and coordinate. Google describes the two as complementary.

Are these ideas new? The LLM versions are new. The core ideas are decades old: blackboard systems such as Hearsay-II were described by 1980, Reid G. Smith's Contract Net Protocol dates from 1980, and agent communication languages such as KQML and FIPA-ACL came out of 1990s research.

What is the biggest risk with autonomous agents? The sources most often name compounding errors, where a small early mistake grows over many unsupervised steps, along with loops, hallucinations and runaway costs. Checkpoints, human review at key decisions, sandboxed testing and good tracing are the standard defenses.

What is an "orchestrator-worker" pattern? A central model breaks a task into subtasks at run time, assigns them to worker models, and combines the results. It is the modern counterpart of the manager and contractors in the Contract Net Protocol.

Conclusion

AI agent orchestration is the practice of getting several AI components, each with a narrower job, to work together on a task none of them handles well alone. Its key ideas go back to the shared blackboards and contract-net bidding of the late 1970s and 1980s. Large language models revived them, and new protocols such as MCP and A2A are now standardizing them. The patterns range from fixed chains and routers to orchestrator-worker setups and fully autonomous agents. The trade-offs are well documented: more parallelism and breadth, paid for with more tokens, more ways to fail, and harder debugging and oversight. The sources agree on the practical lesson: start simple, measure carefully, give each agent a clear job, and keep humans able to see and steer what the system is doing.

Sources and further reading

• Wikipedia, "Multi-agent system": https://en.wikipedia.org/wiki/Multi-agent_system
• Wikipedia, "Contract Net Protocol": https://en.wikipedia.org/wiki/Contract_Net_Protocol
• Wikipedia, "Blackboard system": https://en.wikipedia.org/wiki/Blackboard_system
• Wikipedia, "Agent Communications Language": https://en.wikipedia.org/wiki/Agent_Communications_Language
• Wikipedia, "AutoGPT": https://en.wikipedia.org/wiki/AutoGPT
• Anthropic, "Building effective agents" (December 19, 2024): https://www.anthropic.com/engineering/building-effective-agents
• Anthropic, engineering post on its multi-agent research system (June 13, 2025): https://www.anthropic.com/engineering/multi-agent-research-system
• Anthropic, Model Context Protocol announcement (November 25, 2024): https://www.anthropic.com/news/model-context-protocol
• Google Developers Blog, Agent2Agent (A2A) protocol announcement (April 9, 2025): https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/
• Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models," arXiv:2210.03629: https://arxiv.org/abs/2210.03629
• Wu et al., "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation," arXiv:2308.08155: https://arxiv.org/abs/2308.08155
• NIST, AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework