From Prompt Engineering to Graph Engineering: The AI Agent Engineering Ladder

Every few months the agent world gets a new "engineering" term. Prompt engineering gave way to context engineering. Then came skills, then agent loops, and now graph engineering. It's easy to write these off as rebranding, but each one marks a real change in where the engineer's effort goes and what the model is trusted to decide on its own. Read in order, they form a ladder. Knowing which rung a problem belongs on is one of the most useful architectural calls you can make.

This post walks up the ladder one rung at a time, then looks closely at the top two: loops and graphs. It covers how they differ, why they belong together, what a mixed harness looks like, and why a modern agent graph is more than a renamed DAG orchestrator.

The AI Agent Engineering Ladder

  [ Graph Engineering   ]   Predetermined, parallel, mixed workflows
            ▲
  [ Loop Engineering    ]   Autonomous tool selection, discovery cycles
            ▲
  [ Skills Engineering  ]   Procedural memory, reusable workflows & SOPs
            ▲
  [ Context Engineering ]   RAG, enterprise data pipelines (CRM, DBs, sheets)
            ▲
  [ Prompt Engineering  ]   Persona control, instructions, "role playing"

One idea runs through the whole post: the rungs stack; they don't replace each other. A graph-engineered system still contains prompts, still assembles context, still loads skills, and still runs loops. Each new rung adds a layer of control on top of the ones below it.

Rung 1: Prompt Engineering

The first rung is about shaping a single model call: give it a persona ("answer as a senior tax accountant"), set a tone, add some few-shot examples, ask for a format. Early on, this was most of the craft, because the model was the whole system.

What it solves: behavior and style for a single turn. Where it breaks: the model only knows what it learned in training. No prompt can tell it about yesterday's order, this customer's contract, or last quarter's numbers. And once a prompt grows past a few hundred lines of rules, it becomes fragile in ways that are hard to test.

Rung 2: Context Engineering

Businesses needed answers grounded in their data, and that created the second rung. Context engineering is the discipline of putting the right information into the context window at the right moment: retrieval-augmented generation over documents, plus live data from CRM records, database rows, tickets, and spreadsheets.

The work moves from wording to plumbing. That means retrieval quality, chunking, ranking, freshness, permissions, and fitting it all into a token budget. (We cover the retrieval side in depth in semantic search, hybrid search, and reranking and Graph RAG.)

What it solves: knowledge. The model can now reason over facts it was never trained on. Where it breaks: context tells the model what is true, not how to do the job. Teams start stuffing procedures into the context window ("when a refund is over $500, first check…"), and every request pays for instructions that only matter some of the time.

Rung 3: Skills Engineering

Skills give the agent procedural memory. A procedure stops being a paragraph re-sent on every call and becomes a packaged capability: a named workflow with its instructions, the tools it needs, and often code that runs deterministically. The agent loads a skill only when the situation calls for it.

This is where standard operating procedures go: how to triage a support ticket, how to reconcile an invoice, how to write a release note in your house style. A skill is closer to a function than a prompt. It has a clear trigger, a clear contract, and it can be versioned and tested on its own.

What it solves: reusable know-how without bloating the context. Where it breaks: a skill encodes a procedure you already know. Plenty of real tasks have no fixed procedure, like "find out why this test is flaky" or "research this vendor." For those, the next step depends on what the last step turned up.

Rung 4: Loop Engineering

Loop engineering hands the model the steering wheel. You define a goal, provide a set of tools (calendar APIs, web search, a GitHub client, a shell), and let the agent run a sense → plan → act cycle. It reads the current state, chooses a tool, looks at the result, and repeats until the goal is met or a budget runs out.

while not done and steps < MAX_STEPS:
    action = llm.decide(goal, history, tools)   # model picks the next step
    if action.type == "finish":
        done = True
    else:
        result = tools[action.name](**action.args)
        history.append((action, result))
    steps += 1

Loops are the right tool for exploratory, open-ended work: fixing a bug, researching an unfamiliar topic, debugging a deployment. In these tasks you can't draw the path ahead of time, because finding the path is the task.

What it solves: autonomy over unknown paths. Where it breaks: loops run one step at a time by nature, so latency adds up with every tool call. They're also non-deterministic, which makes them hard to audit. And they're wasteful on work whose shape you already know. If a morning briefing always means "check GitHub, check the calendar, check the news," paying a model to rediscover that plan every morning, one call after another, is pure overhead.

Rung 5: Graph Engineering

Graph engineering is what happens when you notice that much of what agents do has a known shape. Instead of letting the model find the next step, the engineer draws the workflow as nodes and edges: these steps, in this order, with these parts running in parallel. Deterministic structure goes around non-deterministic parts.

The graph handles the parts of the workflow that are predictable. The model handles the judgment calls inside each node. What you get is predictability, parallelism, and observability, without losing the language understanding that made LLMs worth using in the first place.

Loops vs. Graphs: The Structural Difference

The whole distinction comes down to one question: who decides the next step: the model at runtime, or the engineer at design time?

The coexistence principle

A common misconception is that graph engineering is the successor to loops and that loops are now legacy. In practice, both live in the same agent harness. The graph handles the big-picture structure. Any single node in that graph can start its own loop to solve a messy sub-task.

Think of a graph as the org chart and loops as the people doing the work. The org chart fixes who hands off to whom and which teams work in parallel. It doesn't tell the engineer on the GitHub team how to investigate a failing build. That part is exploratory, so it belongs in a loop.

Case Study: A Mixed Graph-Loop Harness

Waku Agent is a local-first, open-source AI assistant framework, and its design shows how the two patterns fit together. Its architecture is a good template for any team building a production assistant.

[Gateway (UI/API)] ──► [Retrieval Gate (Memories/Skills)] ──► [Graph Workflow Engine]
                                                                       │
                                                        ┌──────────────┴──────────────┐
                                                        ▼                             ▼
                                                [Parallel Node A]             [Parallel Node B]
                                                (GitHub loop)                 (Calendar check)
                                                        │                             │
                                                        └──────────────┬──────────────┘
                                                                       ▼
                                                            [Information Synthesis]
                                                                       │
                                                                       ▼
                                                                [Final Reply]

System topology

• Gateway layer. Takes in user input from a chat UI, an API, or messaging channels like Discord, Telegram, or WhatsApp, and turns it into one request format.
• Retrieval gate. Decides which kinds of memory the request needs: procedural (skills), semantic (lasting facts), or episodic (dated history). It loads only those. This is rungs 2 and 3 working as a single gate. (For more on memory types, see agent memory is a distributed systems problem.)
• Graph workflow engine. Sends the request into a predefined directed acyclic graph, such as a /triage workflow or a /gather workflow, and runs its nodes in dependency order.

Workflow 1: the triage graph

A router node first classifies the request's intent, and that decides how much compute the rest of the run needs. While it does, the graph already starts cheap, predictable lookups, such as a calendar check, in parallel. The router's output then picks an edge. A simple question goes to a fast, cheap model for a one-shot answer. A complex one goes to a larger model running a full tool-use loop. The graph keeps the costs predictable, and the loop only runs when it's actually needed.

Workflow 2: the gather graph

For a briefing like "what do I need to know this morning?", the graph fires every source node at once: open GitHub pull requests, local semantic memory, the calendar, and a live web search. Each finishes at its own pace, and a single synthesis node waits for all of them before merging the results into one answer.

The latency gain is easy to see. A loop would call those four tools one after another, so you'd wait for the sum of their times. The graph waits only for the slowest one. In code, the core of it is small:

async def gather_briefing(user):
    # Fan out: independent nodes run concurrently
    prs, memories, events, news = await asyncio.gather(
        github_node(user),        # may run its own tool-use loop internally
        memory_node(user),
        calendar_node(user),
        search_node(user),
        return_exceptions=True,   # one slow or failed source shouldn't sink the briefing
    )
    # Fan in: one LLM node synthesizes whatever came back
    return await synthesize_node(prs, memories, events, news)

Look at the comment on github_node: a node in a graph can be a whole agent loop. That's the coexistence principle in one line.

"Isn't This Just a DAG?"

Skeptics call graph engineering a rebrand of classic DAG orchestrators like Apache Airflow or AWS Step Functions. The comparison is fair as far as structure goes: nodes, edges, dependency order. But three differences change how you design and run these systems.

1. Non-deterministic nodes. In a traditional data pipeline, each task is deterministic code: same input, same output. In an agent graph, nodes are often LLM calls that interpret natural language, reason about structure, and make subjective formatting choices. A node can behave differently on identical inputs, so the graph has to be built to tolerate and contain that variance.
2. Model-driven edge selection. Classic DAG branching is a hardcoded boolean. Agent graphs can put a model at the branch point to judge the situation at runtime. Is this request simple enough for a fast, cheap model, or does it need a heavyweight loop? The edges are fixed at design time, but which one gets taken can be decided by intelligence, not just an if statement.
3. Agentic guardrails. Because nodes can misbehave in ways deterministic code can't (hallucinating, going off-schema, following injected instructions), agent graphs build in real-time checks, validation, and feedback loops around nodes. Data-pipeline orchestrators never needed that. (See input and output guardrails for the patterns.)

Choosing the Right Rung

In practice, the question isn't which rung is best. It's what the lowest rung is that reliably solves the problem in front of you. Go higher only when the current rung is the actual bottleneck.

A practical habit: start with a loop, watch it run, then build the graph from what you saw. When traces show the agent taking the same path on 80% of requests, that path is a graph waiting to be drawn. Make it fixed and parallel. Leave the other 20% to a loop inside a fallback node. (Our post on measuring agent quality covers the tracing and evaluation that make this habit possible.)

Closing Thoughts

The move toward graph engineering is a big step in making agent workflows stable enough for business use. Wrapping autonomous loops inside deterministic graph shapes gives teams predictable paths, lower latency from parallel nodes, and solid operational control over parts that will never be fully deterministic.

But the ladder is cumulative. The best graph is still only as good as the prompts in its nodes, the context it pulls in, the skills it loads, and the loops it hands open-ended work to. Loops and graphs aren't rival philosophies. They're two tools in one harness: graphs for the parts you can predict, loops for the parts you can't.

Related reading:
AI agents are just backend microservices · Agent memory is a distributed systems problem · Graph RAG beyond single-hop retrieval · Input and output guardrails in AI agents · How to evaluate AI agents

← Back to all posts