August 8, 2026

From Single-Prompt to Multi-Agent: Orchestrating Local LLMs with a Centralized Controller

A practical architectural guide to converting a simple LLM interaction into a multi-agent system with a centralized orchestrator that dynamically spawns, delegates to, and terminates specialized worker agents—all running locally with open-source models.

The Problem

The simplest form of LLM integration is a single prompt–response loop: the user sends a message, the model returns a response, and the transaction ends. This works for trivial Q&A. It fails—silently and catastrophically—when the task requires multiple distinct reasoning paths, verification of intermediate outputs, or context that exceeds a single prompt’s carrying capacity.

Consider a realistic enterprise scenario: a user asks an AI system to “research the competitive landscape of edge computing platforms, write a technical brief, and ensure accuracy.” A single-prompt approach forces one model to simultaneously be a researcher, writer, and fact-checker. The model attempts all three roles with a single monolithic system prompt, leading to diluted attention, inconsistent quality, and no mechanism to verify its own claims. There is no separation of concerns, no iterative refinement, and no way to parallelize independent workstreams.

The industry’s response to this limitation has been to throw larger models at the problem. But research consistently shows that architectural improvements in the surrounding infrastructure outperform raw model upgrades by a significant margin. A well-orchestrated system of small, specialized models can outperform a single frontier model operating without structure.

The Architectural Solution

The Orchestrator-Worker pattern decomposes a complex task into a centralized coordination layer and a fleet of stateless, specialized agents. The architecture is modeled on a technical lead delegating to individual contributors—each with a narrow scope, a clear deliverable, and no knowledge of other workers.

The system has three core components:

Orchestrator Agent (The Brain): A coordination-only agent backed by the most capable available model. It owns the user intent, maintains the full conversation state, and makes delegation decisions. Critically, the orchestrator never performs the actual work—its only “tool” is calling other agents.
Worker Agents (The Specialists): Narrow-scope agents with focused system prompts and restricted responsibilities. Each worker receives a specific sub-task, executes it independently, returns the result to the orchestrator, and terminates. Workers have no awareness of each other and no persistent state across invocations.
Shared State (The Blackboard): A conversation history maintained exclusively by the orchestrator. After each worker returns, the orchestrator appends the result to this shared state before deciding the next action. This provides context continuity without requiring workers to share memory.

Centralized Multi-Agent Orchestrator Blueprint

The execution follows a deterministic loop:

Orchestrator receives user request
  → Delegates to Worker A (e.g., "research WebSockets")
  → Worker A returns findings, terminates
  → Orchestrator receives findings, delegates to Worker B (e.g., "write article")
  → Worker B returns draft, terminates
  → Orchestrator receives draft, delegates to Worker C (e.g., "review quality")
  → Worker C returns verdict, terminates
  → If verdict = "pass" → Orchestrator returns final output
  → If verdict = "revise" → Orchestrator re-delegates to Writer with feedback

Why Not a Single Smart Model?

The multi-agent decomposition provides structural advantages that no single model—regardless of parameter count—can replicate:

Prompt isolation: Each worker’s system prompt is narrow and focused. A researcher’s prompt says “gather facts, do NOT write articles.” A writer’s prompt says “transform research into prose, do NOT invent facts.” This constraint enforcement is impossible in a monolithic prompt without dilution.
Context efficiency: The orchestrator passes only relevant context to each worker—not the entire conversation history. A reviewer receives the draft article, not the raw research output. This prevents the context rot described in prior harness engineering analysis.
Verification gates: A separate reviewer agent provides an external quality gate. When a model evaluates its own output, it exhibits confirmation bias. When a separate agent with a separate system prompt evaluates the output, it provides genuinely independent assessment.
Resource optimization: Workers can run on smaller, faster models (3B–8B parameters) because their tasks are narrowly scoped. The orchestrator, which requires reasoning and planning, benefits from a larger model. This is a heterogeneous compute strategy: allocate intelligence where it is needed.
Parallelism: Independent sub-tasks (e.g., “research topic A” and “research topic B”) can be dispatched concurrently using asynchronous execution, reducing total wall-clock time.

Core Implementation

The following is a complete, functional implementation of the orchestrator-worker pattern using Ollama as the local LLM runtime. Ollama exposes an OpenAI-compatible API for any open-source model, eliminating cloud API costs and data privacy concerns.

Prerequisites:

pip install ollama
ollama pull llama3.2        # Worker model (fast, 8B)
ollama pull llama3.2:70b    # Orchestrator model (smart, 70B)
# If 70B is too large for local hardware, use llama3.2 for both roles.

Agent Definitions:

The agent registry defines each worker’s identity, system prompt, and model assignment. Workers are stateless data classes—they are instantiated per-task and discarded after returning a result.

import json
import ollama
from dataclasses import dataclass, field
from datetime import datetime


@dataclass
class AgentConfig:
    """Defines a specialized worker agent."""
    name: str
    role: str
    system_prompt: str
    model: str = "llama3.2"


AGENT_REGISTRY: dict[str, AgentConfig] = {
    "researcher": AgentConfig(
        name="researcher",
        role="Research Specialist",
        system_prompt=(
            "You are a Research Specialist. Your ONLY job is to gather and "
            "summarize factual information about the topic you are given. "
            "Be thorough but concise. Output ONLY the research summary. "
            "Do NOT write final articles, do NOT give opinions."
        ),
    ),
    "writer": AgentConfig(
        name="writer",
        role="Technical Writer",
        system_prompt=(
            "You are a Technical Writer. You receive research summaries and "
            "your ONLY job is to transform them into a well-structured, "
            "engaging article. Use clear headings, bullet points where "
            "appropriate, and a professional tone. Output ONLY the article."
        ),
    ),
    "reviewer": AgentConfig(
        name="reviewer",
        role="Quality Reviewer",
        system_prompt=(
            "You are a Quality Reviewer. You receive a draft article and "
            "your ONLY job is to evaluate it for: (1) factual accuracy, "
            "(2) clarity, (3) completeness, and (4) tone. Output a JSON "
            'object: {"verdict": "pass" | "revise", "feedback": "..."}. '
            "Be strict but fair."
        ),
    ),
}

Worker Execution:

The spawn_worker function creates a one-shot interaction with a worker agent. The worker receives its system prompt and task, generates a response, and returns it. No state persists beyond this single call—the worker is effectively “terminated” after returning.

@dataclass
class WorkerResult:
    agent_name: str
    output: str
    timestamp: str = field(
        default_factory=lambda: datetime.now().isoformat()
    )


def spawn_worker(config: AgentConfig, task: str) -> WorkerResult:
    response = ollama.chat(
        model=config.model,
        messages=[
            {"role": "system", "content": config.system_prompt},
            {"role": "user", "content": task},
        ],
    )
    return WorkerResult(
        agent_name=config.name,
        output=response["message"]["content"],
    )

Orchestration Loop:

The orchestrator’s system prompt instructs it to respond exclusively in structured JSON, declaring either a delegate action (with a target agent and task) or a finish action (with the final answer). Ollama’s format="json" parameter constrains the model output to valid JSON syntax.

ORCHESTRATOR_PROMPT = """You are a Task Orchestrator managing specialist agents.

Available agents: {agents}

Respond with ONLY valid JSON:
To delegate: {{"action": "delegate", "agent": "<name>", "task": "<description>"}}
To finish:   {{"action": "finish", "answer": "<final answer>"}}

Rules:
- Call agents one at a time: researcher, then writer, then reviewer.
- If reviewer says "revise", call writer again with the feedback.
- Maximum 2 revision cycles, then finish with the best version.
"""

MAX_TURNS = 10


def orchestrate(user_request: str) -> str:
    agent_list = ", ".join(
        f'"{n}" ({c.role})' for n, c in AGENT_REGISTRY.items()
    )

    conversation = [
        {"role": "system",
         "content": ORCHESTRATOR_PROMPT.format(agents=agent_list)},
        {"role": "user",
         "content": f"User request: {user_request}"},
    ]

    for turn in range(MAX_TURNS):
        response = ollama.chat(
            model="llama3.2",
            messages=conversation,
            format="json",
        )
        raw = response["message"]["content"]

        try:
            decision = json.loads(raw)
        except json.JSONDecodeError:
            conversation.append({"role": "assistant", "content": raw})
            conversation.append({
                "role": "user",
                "content": "Invalid JSON. Respond with ONLY a JSON object."
            })
            continue

        action = decision.get("action")

        if action == "finish":
            return decision.get("answer", "No answer provided.")

        if action == "delegate":
            agent_name = decision.get("agent", "")
            task = decision.get("task", "")

            if agent_name not in AGENT_REGISTRY:
                conversation.append({"role": "assistant", "content": raw})
                conversation.append({
                    "role": "user",
                    "content": f"Unknown agent '{agent_name}'. "
                               f"Available: {agent_list}"
                })
                continue

            result = spawn_worker(AGENT_REGISTRY[agent_name], task)

            conversation.append({"role": "assistant", "content": raw})
            conversation.append({
                "role": "user",
                "content": f"Result from {result.agent_name}:\n\n"
                           f"{result.output}"
            })

    return "Orchestrator exceeded maximum turns."

Production Hardening

Moving from a proof-of-concept to a production system requires addressing failure modes that do not appear in happy-path testing:

Infinite loop prevention. The MAX_TURNS constant is the primary safety valve. Without it, an orchestrator that receives a “revise” verdict from the reviewer can enter an unbounded write-review-revise cycle. In production, set this conservatively and log when the limit is reached.

Structured output enforcement. Ollama’s format="json" guarantees syntactically valid JSON but does not enforce schema compliance. For production, define a Pydantic model for the orchestrator’s response and validate with model_validate_json() after every turn. This catches cases where the model returns valid JSON with unexpected field names.

Worker isolation and error handling. Wrap spawn_worker in a try/except block. If a worker fails (model timeout, OOM, corrupted output), the error should be reported back to the orchestrator as a structured message: "Result from researcher: ERROR — Model timed out after 120 seconds." The orchestrator can then decide whether to retry or route to an alternative agent.

Context window management. As the conversation grows across turns, the orchestrator’s context approaches its window limit. For long-running workflows, implement context compaction between turns: summarize older worker results into compact representations and pin the original user request and current state variables.

Observability. Log every turn with: turn number, action taken, target agent, task description (truncated), result length, and wall-clock time. This trace is essential for debugging delegation failures and performance regression.

Edge Cases & Limitations

Multi-agent orchestration is not universally superior. It introduces latency (each delegation is a full model inference), token overhead (the orchestrator consumes tokens on every turn), and operational complexity (more moving parts to monitor and debug).

Use a single model when:

The task can be fully expressed in one well-crafted prompt
Latency is the primary constraint
The task does not require verification or iterative refinement

Use multi-agent orchestration when:

The task requires multiple distinct reasoning skills
Intermediate outputs must be verified by an independent evaluator
Different sub-tasks benefit from different model sizes or specializations
The full problem context exceeds a single model’s effective attention span
You need an audit trail of delegation decisions for compliance

Framework Landscape

For teams that need more than the raw Python loop shown above, the following frameworks provide mature multi-agent orchestration with local LLM support via Ollama’s OpenAI-compatible API:

LangGraph — The current industry standard for stateful, cyclic agent workflows. Best for complex branching logic where agents pass state back and forth through a directed graph.
AG2 (formerly AutoGen) — Strong for conversational patterns where agents “debate” a solution. Well-suited for code generation and review workflows.
LlamaIndex Workflows — Purpose-built for RAG-heavy pipelines where agents need to search, retrieve, and synthesize from vector stores.
CrewAI — Highest-level abstraction for role-based team simulations. Fast to prototype but less flexible for custom control flow.

The raw Python approach demonstrated in this article provides maximum control and transparency. It is the recommended starting point for understanding the mechanics before adopting a framework that abstracts them away.

Key Strategic Takeaways

Decouple Coordination from Execution: The orchestrator must strictly govern flow, context routing, and verification without performing raw generation work.
Heterogeneous Compute Allocation: Optimize resource efficiency by deploying compact, specialized models (3B–8B) for bounded worker tasks while reserving frontier/large models for orchestrator planning.
Independent Verification Eliminates Confirmation Bias: An external reviewer agent with a dedicated adversarial prompt catches hallucinations that a monolithic prompt would miss.