AAOSA: A Distributed Orchestration for Multi-Agent Systems

AAOSA introduces a distributed orchestration model where independent agents evaluate, claim, and coordinate tasks to collectively interpret and respond to complex inputs.

Key Takeaways

• AAOSA (Adaptive Agent-Oriented Software Architecture) is a distributed orchestration mechanism for multi-agent systems.

• Instead of a centralized controller, individual agents evaluate user input and claim responsibility for the parts they can handle.

• This architecture enables scalable coordination across large agent networks while preserving modularity.

• AAOSA was originally developed for natural-language command systems and influenced early implementations of Siri.

• Today, it provides a foundation for orchestrating networks of LLM-powered agents.

Many years ago, while I was looking for real world problems to showcase the power of distributed AI, I was challenged to tackle natural language command and control systems. At the time, the complexities of understanding natural language made this challenge seem daunting.

Natural language is ambiguous, contextual, and often ungrammatical. Traditional centralized parsing approaches struggle when the domain grows or when commands span multiple subsystems. It was not obvious how to scale such a system while preserving modularity.

Still, I kept coming back to a simple idea, one that felt almost too straightforward to work: what if the elements of the domain we were trying to control could look back at the user’s command and stake a claim on the parts they believed they could handle? 

Instead of a single component trying to understand everything, each specialized component would evaluate the command from its own perspective. In doing so, it elegantly retained the encapsulation of responsibilities while allowing for a highly modular breakdown of natural language processing. Each agent focused only on what it knew how to handle. Understanding emerged from distributed evaluation rather than centralized interpretation.

This was the foundation of what later became Adaptive Agent-Oriented Software Architecture, or AAOSA.

What Is AAOSA?

Adaptive Agent-Oriented Software Architecture (AAOSA) is a distributed orchestration mechanism for multi-agent systems.

Instead of relying on a single central orchestrator to interpret user input and delegate tasks, AAOSA allows each agent in the system to evaluate incoming commands and determine whether it can contribute to fulfilling them.

Agents effectively “claim” responsibility for the parts of a command that match their capabilities. These claims are then consolidated across the system to produce a coordinated response or set of actions.

This design enables systems that are:

• Highly modular

• Extensible

• Resilient to ambiguity

• Scalable to large numbers of agents

In AAOSA, understanding is not produced by a central parser but emerges from the collective evaluation of specialized agents.

Origins of AAOSA: A Distributed Agent Home Command System

We put this idea into practice in 1997 by building a natural language command-and-control system for a home entertainment setup.

The system was organized as a hierarchy of AI agents. The user is communicating to the top agent, which has some down-chain agents responsible for various appliances (e.g., TV, VCR, DVD, Satellite, lights...). Each of these agents, in turn, may have down-chains of their own. For example, the VCR may have down-chain agents like Play, Stop, Rewind, Forward, and Record). 

To ensure full encapsulation, agents only knew what their down-chains reported to them, along with the set of tools or APIs they were responsible for and some examples of how people might refer to them. The Play agent, for example, knew that words and phrases like “play,” “show,” or “I want to see” directly referred to what it could do. 

Example: Distributed Command Handling

When a command like 'Play the VCR and dim the lights' is issued, the top agent did not attempt to fully interpret the sentence. Instead of refusing the command, it passed the request down the hierarchy. It checks with its down-chain agents, who in turn, check with their down-chain agent. Ultimately, the leaf agents controlling the functionality intended make the strongest claims and get to actuate on them in a second pass.

Theoretical Foundations

The approach had some solid theoretical foundation too. I proved the soundness and correctness of the approach as part of my PhD (Applying the Adaptive Agent Oriented Software Architecture to the Parsing of Context Sensitive Grammars, Babak Hodjat and Makoto Amamiya, IEICE Trans. Vol.E83-D, No.5, pp.1142-1152, May 2000).

While the agents of those days, being rules-based with hardcoded logic, were far less capable than the ones today, the resulting system, which ended up being used in the early implementations of Siri, proved to be quite robust. It was more or less language independent, could handle non-grammatical commands, allowed for an extensible system where agents could be added or removed without the need for reengineering everything, and it handled ambiguities and incorporated context naturally.

We called this distributed agentic coordination mechanism Adaptive Agent-Oriented Software Architecture (AAOSA), and we thought of it as the necessary ingredient for extensible Agent-Oriented systems (as opposed to its more rigid Object-Oriented predecessor).

Why Distributed Orchestration Matters in Modern Multi-Agent Systems

Today, agents are much more powerful. Each agent can leverage a large language model for reasoning, generation, and tool use. The granularity required in the 1990s is no longer necessary. 

There is, however, still a need for distributed orchestration to facilitate extensible and modular multi-agent systems and avoid the concentration of responsibilities and the inevitable risks associated with that.

Let’s say, we have an agent helping our enterprise users with some business knowledgebase. Let's also say that in addition to the original agent, another agent is introduced into our business that, on behalf of users, operates another function, say a micro-service, and some of the user’s queries or commands are better handled through that service, or a combination of the two. We have a choice of forcing the user to decide which agent to use and then consolidating the responses, or to have one of the two agents collaborate with the first agent that is in direct interaction with the user —making the user experience more seamless and usable.

To make this happen, we can have the original agent servicing the user aware of the agent wrapping the microservice and have them communicate to fulfill user queries. This communication can be in natural language, which means it can capture the intent and be robust to changes in the format and syntax of the specific calls to the underlying services. This ultimately allows upgrades to the underlying systems with minimal impact on the overall user experience.

It is clear to see how this approach can lead to an entire software architecture composed of agents representing various functions communicating with one another, or with human users.

A multi-agent system will require a coordination mechanism to identify agents that are responsible for responding to input. In its simplest form, a coded mechanism can decide which agent(s) to invoke. For example, a ‘coordinator’ agent will have a set of rules that determine which agents to call, and in what order.

Many multi-agent systems today are limited to a single 'orchestrator' agent divvying up tasks between a hand-full of agents. In some cases, a classifier is employed to decide which subset of agents should be invoked. This classifier may be ML-based, trained on historical labeled samples.

This does not scale. Agents have a limit on their context as well as their tool use abilities. Also, more specialized agents can be provisioned from different vendors, or be customized for specific apps or data sources. Our agentic future will require the coherent orchestration of hundreds, if not thousands of agents.

Related Architectures

In larger and more complex multi-agent systems, the agents themselves encapsulate and determine the logic for whether they can play a role in responding to a query or command. For example, in blackboard architectures (e.g., Open Agent Architecture or OAA), all agents register with a blackboard, advertising what they can do, and what requirements they would need to do it. Upon receiving a query or command, the blackboard decides which agents to call, and in what order.

Note that it is crucial to make sure an agent's LLM world knowledge takes a back seat if there are custom context or tools available. This is because the agent's world knowledge is general, gets stale quickly, is more subject to confabulation, and is typically interpolated from a vast training corpus.

Designing Safe Multi-Agent Systems

Multi-agent systems need to be engineered and designed for safety, reliability, and trustworthiness. Modularity is the essential ingredient of engineering, and we can think of agents as Lego blocks for building businesses of the future. These Lego blocks need to come together coherently and effectively, and so a distributed orchestration and coordination mechanism is needed.

AAOSA is very much applicable in modern day multi-agent systems. It guarantees encapsulation, modularity, and extensibility, and it also elegantly handles the consolidation of responses and actuations across the multi-agent system so that a unified response is generated for commands and contexts.

Here's how this consolidation works: In AAOSA, each agent is an orchestrator for its down chains, which means it is tasked with retrieving claims from its down chain and deciding how to put them together in a coherent response up to its up-chain agent. This decision may include disambiguation of conflicting claims, which may require contextual information, or interaction with the user. This, in turn, means that AAOSA has the capacity to consolidate UI elements as well. This is the basis for the Context Reactive User Experience (CRUSE), where each agent provides UI snippets relevant to its role in handling a command or a context, and these snippets are consolidated by the up-chains all the way back to the user.

Entry-Point Agnostic Orchestration

AAOSA is also entry-point agnostic, in that any agent can be thought of as the top agent, and it will be able to orchestrate and coordinate responses from all agents within its hierarchy. In other words, as a user, if you know which agentic sub-system is best suited for your needs, you can command or query that sub-system without having to start from the top orchestrator agent every time and incur the cost and response time for doing so.

AAOSA in neuro-san

We have implemented AAOSA as a default orchestration mechanism within our open-sourced agentic framework neuro-san. The current implementation is through prompts and the agents run AAOSA in a semi-structured claim and actuation language. This has made the building of large and powerful agent networks quite seamless and is part of what allows us to vibe large, hierarchical agent networks into existence and run them out of the box without worrying about the orchestration details.

For more details and to see AAOSA in action, go to: https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/docs/user_guide.md#aaosa

Conclusion

AAOSA demonstrates how distributed orchestration can provide a scalable foundation for multi-agent systems.

By allowing agents to claim responsibility for the parts of a command they understand, the system achieves:

• modularity

• extensibility

• scalable coordination

Originally developed for natural language command systems, AAOSA remains highly relevant today as organizations begin deploying large networks of LLM-powered agents across enterprise environments.

Distributed orchestration will be a key ingredient in building the agent-driven software architectures of the future.