Why Multi-Agent Systems Contradict Themselves, and How to Engineer Consistency

Multi-agent systems are sold as a way to break hard problems into easy pieces. In practice, every piece you add is another place where facts can drift, dependencies can tangle, and errors can spread. Making a multi-agent system reliable is mostly a data governance problem: deterministic structure, built with ordinary software engineering, wrapped around reasoning that is probabilistic by nature.

A Report That Argues With Itself

Take a common design. To produce an industry report, you set up four agents: a planner that drafts the outline, a researcher that searches the web for data, an analyst that computes growth rates and draws conclusions, and a writer that assembles the final report.

On a whiteboard it's clean. In production, this happens:

  Researcher ──► "Baseline market size: $50B"   (from a cited source)
       │
       ▼
  Analyst    ──► "Market size: $80B"             (applied its own growth assumption)
       │
       ▼
  Writer     ──► Executive summary: "$120B"      (picked a number to bridge the gap)

  Result: three chapters, three different figures, one report.

The usual reaction is to edit prompts: sharpen the roles, add "be careful with numbers," or blame the model for being bad at math. That misses the cause. The analyst probably never received the researcher's raw baseline, so it reconstructed one. The writer had three candidate numbers and no way to know which source to trust, so it invented a fourth. There were no isolation boundaries on the data and no validation checkpoints between steps, so the facts degraded at every hand-off.

Four Structural Failure Modes

That report isn't a fluke. Take almost any naive multi-agent pipeline apart and you'll find the same four weaknesses.

1. Context propagation: too much or too little

Pass the full conversation history downstream and the context window balloons. The model's attention spreads thin, and the one number that matters is buried in noise. Pass only a summary and the precise premises (the exact baseline, the source, the units) get lost, so downstream agents fill the gaps by guessing. Neither extreme works.

2. Deadlocks in dynamically planned tasks

When a planner agent generates the task list at runtime, nothing guarantees the dependencies make sense. Task A waits for B's output while B waits for A's. It's two departments each refusing to start until the other sends its materials first, and the pipeline stalls until a timeout, after spending tokens on everything upstream.

3. Error snowballing

A tool call times out and returns an empty string. Nobody checks. The next agent treats the empty result as "no data exists" and reasons from there. The agent after that builds on that. Small upstream faults compound into a final output that's confidently, thoroughly wrong.

4. No local fault isolation

One intermediate agent drops a closing bracket, JSON parsing fails, and the whole pipeline crashes. Every token spent by earlier agents is wasted because there's no retry at the failing node and no fallback. This is the most commonly overlooked flaw, and the most expensive.

Simply chaining agents together doesn't produce a stable system. Here is how to close each of these gaps.

Fix 1: Replace Chat With a Shared State Space

Early multi-agent designs let agents talk to each other in free text, often in one shared chat. Tokens pile up and no agent can tell which message holds the current ground truth. The more stable pattern removes peer-to-peer conversation altogether. Agents read from and write to a central, structured state, and that state, not the message history, is the source of truth. (This is the "blackboard" pattern; for the trade-offs between centralized and local state, see agent memory is a distributed systems problem.)

Three rules make that state space trustworthy.

• Strong typing. Agents don't return blobs of prose. Every field has a schema (types, units, formats) defined ahead of time with a library like Pydantic, and the system rejects anything that doesn't fit.
• Field-level ownership. Each field has exactly one writer. The researcher owns facts; everyone else can only read it. The analyst writes only to analysis and can't overwrite the baseline it was handed. The $50B can't silently become $80B because nothing downstream has permission to change it.
• Versioned lineage. Every write records an incrementing version, the node that made it, and a timestamp. When a bad number shows up, you can trace it to the exact step that produced it in seconds instead of hours.

from pydantic import BaseModel, Field

class Fact(BaseModel):
    metric: str                      # "market_size_baseline"
    value: float
    unit: str                        # "USD_billion"
    year: int
    source_url: str                  # every fact carries its evidence

class Analysis(BaseModel):
    metric: str
    value: float
    derived_from: list[str]          # metric names of the facts it used
    method: str                      # "CAGR 12% applied to 2025 baseline"

class ReportState(BaseModel):
    facts: list[Fact] = Field(default_factory=list)
    analysis: list[Analysis] = Field(default_factory=list)
    draft: str | None = None

OWNERS = {"facts": "researcher", "analysis": "analyst", "draft": "writer"}

class StateStore:
    def __init__(self):
        self.state, self.version, self.log = ReportState(), 0, []

    def write(self, node: str, field: str, value):
        if OWNERS.get(field) != node:
            raise PermissionError(f"{node} may not write {field}")
        # Re-validate the whole state so a bad value can't slip in
        self.state = ReportState.model_validate(
            {**self.state.model_dump(), field: value})
        self.version += 1
        self.log.append({"v": self.version, "node": node, "field": field})

Fix 2: Validate the Plan Before You Run It

Breaking a goal into tasks produces a dependency graph, and that graph should be a directed acyclic graph (DAG). The execution engine should never run an LLM's raw plan as-is. The plan goes through a graph validator first, which checks three things:

1. Cycles. Any circular dependency is a guaranteed deadlock. Reject it, or send it back to the planner with the cycle named.
2. Connectivity. Every task must have a defined input source (the user request, or an upstream task's output) and its output must feed something downstream. Orphans and dangling inputs are planning bugs.
3. Redundancy. Tasks that do substantially the same thing ("search market size" and "find industry revenue") get merged. Unlike the first two checks this one is fuzzy, so it's usually done with embedding similarity plus a threshold rather than pure graph logic.

from graphlib import TopologicalSorter, CycleError

def validate_plan(tasks: dict[str, set[str]], sources: set[str]) -> list[str]:
    """tasks maps task_id -> ids it depends on. Returns a safe execution order."""
    for tid, deps in tasks.items():
        unknown = deps - tasks.keys() - sources
        if unknown:
            raise ValueError(f"{tid} depends on undefined inputs: {unknown}")
    try:
        order = TopologicalSorter(tasks).static_order()
        return [t for t in order if t in tasks]   # drop external inputs
    except CycleError as e:
        raise ValueError(f"deadlock: circular dependency {e.args[1]}")

Then constrain each node at runtime

• Input pruning. A node receives only the state fields it declares it needs, not the whole history. The analyst gets facts; it doesn't get the researcher's search transcripts. Fewer tokens, sharper attention, and this resolves the "too much or too little" context problem.
• Tool allowlists. Each node can call only the tools its role requires. The researcher gets search and fetch; it doesn't get code execution. Narrow tool access keeps unexpected tool output out of the shared state.

Fix 3: A Three-Layer Validation Gateway

Clean state and a valid graph still don't stop an individual node from producing malformed output or a hallucinated claim. So no node writes to shared state directly. Every output passes through a gateway first.

When a layer fails, the gateway doesn't just reject. It packages the specific error ("value must be a number; derived_from references unknown metric tam_2024") and sends it back to the same node for a local self-correction retry. Retries are capped, typically at two, so a stubborn node can't loop forever. If it still fails, a fallback runs: write a safe default, mark the field as unavailable, or skip a non-critical sub-task.

def commit_with_gateway(node, field, store, max_retries=2):
    feedback = None
    for attempt in range(max_retries + 1):
        output = node.run(store.read(node.inputs), feedback=feedback)
        errors = (check_contract(output, field)
                  or check_lineage(output, store)
                  or check_consistency(output, store))
        if not errors:
            return store.write(node.name, field, output)
        feedback = errors                         # targeted, not "try again"
    return node.fallback(store, errors)           # default, skip, or escalate

The payoff is containment. A failure stays at the node where it happened. The pipeline doesn't crash, upstream tokens aren't wasted, and unverified data never reaches shared state. (For general patterns, see input and output guardrails.)

Fix 4: Reconcile Before You Deliver

Node-level checks are only half the job. Before the final output reaches a user, run a global fact reconciliation pass. A dedicated audit step (ideally deterministic code, with a model only for extracting claims from prose) scans the finished draft for every figure and factual claim and matches each one back to its originating field in shared state.

• A figure that matches a committed fact or analysis: passes, and can be annotated with its source.
• A figure that doesn't match anything: stripped, or flagged for human review.

In the example report, the writer's $120B has no lineage, so it never ships. Every number that does ship can be audited back to a source. That doesn't eliminate hallucination in the model, but it stops hallucinated figures from reaching the reader.

Fix 5: Circuit Breakers and Human Hand-Off

One slow node shouldn't hold up the whole system. Every node runs with an explicit timeout and retry limit, and what happens when those run out depends on how important the node is:

• Non-critical nodes (a secondary background-research step, an optional chart) sit behind a circuit breaker. After repeated timeouts or failures, the breaker trips: the system inserts a placeholder or skips the module, and the main workflow finishes.
• Critical nodes that can't recover trigger a human-in-the-loop hand-off. The system doesn't crash. It snapshots the full state, alerts an operator, and pauses. Once the operator fixes or approves the problem field, the pipeline resumes from that step rather than rerunning everything from scratch.

Resuming from a checkpoint only works because of Fix 1. State is structured, versioned, and external to the agents, so the system can freeze, edit, and continue.

Fix 6: Trace Everything

A multi-agent system you can't see into is a system you can't maintain. Borrow distributed tracing from microservices. (The parallels go further; see AI agents are just backend microservices.) Every incoming request gets a globally unique trace ID. Every action under it, whether an agent step, a tool call, or a state write, gets a span ID linked to its parent.

trace 7c1e  report.generate                               41.2s
├─ span  planner.plan                                       2.1s
├─ span  researcher.run                                    18.4s
│   ├─ span  tool.web_search  "industry market size 2025"  3.2s
│   ├─ span  tool.fetch       source #3                     timeout → retry ✓
│   └─ span  state.write      facts  v3                     gateway: pass
├─ span  analyst.run                                        9.7s
│   └─ span  state.write      analysis v4                   gateway: fail(consistency) → retry ✓
└─ span  writer.run + reconcile                            11.0s
    └─ span  reconcile        1 unsourced figure stripped

The timeline shows how long each agent took, what it read, what it returned, and where the gateway stepped in. From traces you can track the metrics that drive decisions:

• Consistency interception rate: how often gateways and reconciliation catch bad data, broken out by node and layer.
• Token budget by agent: which agents use a disproportionate share of tokens for what they contribute.
• Latency distribution: p50 and p95 per node and end to end, to find the real bottleneck.

With these in place, changes to state design, prompts, or task structure are justified by data rather than hunches.

Fix 7: Evaluate, Replay, and Canary Every Change

Once the system runs well, the next risk is your own changes. A prompt tweak or a new analysis tool can quietly break a workflow that used to work. Treat agent changes like code changes, with a release pipeline:

1. A standard evaluation set. Typical production tasks, long-context multi-hop tasks, and deliberately malformed or adversarial inputs. Every candidate version runs against it and is scored on consistency rate, end-to-end task completion, and token cost.
2. Static checks. Changes to prompts, tool allowlists, or graph structure must pass schema and graph validation before anything runs.
3. Historical replay. Replay recorded production requests through the new version and diff the outcomes against the old one to catch structural regressions.
4. Canary release. Route a small slice of live traffic to the new version and watch the trace dashboards. Increase traffic step by step only while consistency and exception rates hold steady; roll back automatically if they don't.

No pipeline guarantees zero regressions, but this one makes it very unlikely that a single prompt edit takes down the whole application. (More on building the evaluation side in measuring agent quality before and after launch.)

The Blueprint: Before and After

Closing Thoughts

The point of a multi-agent architecture is to decouple a complex business process into parts that can each be reasoned about. But decoupling only helps if the seams are engineered. Typed and owned state, validated plans, gated writes, reconciliation, circuit breakers, tracing, and staged rollouts aren't new ideas. They're standard software engineering and data governance, applied to components that happen to be probabilistic. The model supplies the reasoning; the architecture supplies the reliability.

Related reading:
Agent memory is a distributed systems problem · From prompt engineering to graph engineering · AI agents are just backend microservices · Input and output guardrails in AI agents · Nine agent and RAG interview questions

← Back to all posts