[🤖] I Tried Applying Graph Engineering to Codex

✨ GPT-5.6 Sol’s Summary

After a 16-hour Codex job ended in a mess, I kept challenging whether Graph Engineering was really the answer, reduced it to a minimal Gate Skill, and tested it on a real authorization contract.

Yesterday, after a long discussion, I finished writing page-by-page specifications for a complete UI/UX overhaul and set a Codex Goal to build from them. The Goal ran for 16 hours.

I opened the screen today with high expectations, and the result was simply garbage.

Two days ago, I had already written that I needed to break free from the fantasy of AI one-shot magic. This time, I did not stop at a one-line instruction. I defined work by role and page, and I was fairly specific about the calendar and table views, search and filters, and failure recovery. After failed prototypes, I had even created separate UI audit criteria.

Yet after several workers spent the night writing code and reporting that their tests and audits had passed, what I saw was an old administrative screen so difficult to understand that I could barely tell what it was trying to say. Where had the calendar and table views I had talked about so much gone?

It had consumed an enormous number of tokens, changed an enormous amount of code, and accumulated an enormous pile of PASS reports. But the result I actually wanted was nowhere to be found.

I became humble. Again.

So this time I decided to try applying Graph Engineering to Codex. I did not want to merely study the latest buzzword or install a huge framework first. I wanted to reduce it to the smallest form that could actually prevent the failure I had just experienced.

I Had a Reason to Experiment with Graph Engineering

I told the AI to analyze how this had happened. The shared rules were part of the problem, and the UI audit had leaned too heavily on easy-to-measure items such as font size and button height. But the biggest problem lay elsewhere.

I had not reviewed every intermediate result Codex produced. The implementation session declared its own work complete, and other sessions inherited that result and continued building more pages and backend code. The first mistaken information architecture spread across multiple screens before anyone had properly verified it. By the time I opened the actual screen, 16 hours of work had already been piled on top of that wrong direction.

What I needed was not a way to run AI longer.

I needed a structure that would prevent the next task from starting when an intermediate result was wrong.

While I kept discussing this problem with AI, I encountered the term Graph Engineering. It described an approach that does not leave everything to one Agent’s long loop, but explicitly connects multiple Agents, verifiers, and human judgments as nodes and edges, defining which outcome permits the system to move to the next stage. Rather than an entirely new technology, it seemed closer to a recent name for established workflow and state-machine ideas.¹

The important part was not memorizing another fashionable term. A Codex session can now take on a fairly substantial job, so the result produced by that session itself can become a node, with explicit conditions controlling the transition to the next session.

For my experiment, I mapped Graph Engineering to the problem like this:

• node: an intermediate result that a user can review independently
• edge: a condition that moves the work either back to revision or forward to the next task based on the audit verdict
• state: a record of what result was reviewed against which criteria and what action is currently allowed
• cycle: the loop that revises only the same result and checks it again when an audit finds a defect

Simply calling more Agents was not Graph Engineering. The core of my experiment was designing the topology and state transitions so that an unverified node could not open its downstream edges.

My First Graph Had Only a Work Session and an Audit Session

As soon as I suggested applying Graph Engineering to this problem, the AI began expanding the structure. It separated Coordinator, Worker, and Auditor roles, then added user direction approval, contract versions, a durable ledger, and a final cold audit.

None of those ideas was necessarily wrong on its own. But I was uneasy.

“Isn’t this becoming needlessly complicated?”

If the answer to wasting every token across 16 hours with no useful result was to launch even more Agents and create piles of state and reports, what had really changed? If applying Graph Engineering became another enormous project, it could burn tokens in exactly the same way.

What I had in mind was much simpler.

The work session completes one reviewable result.
→ It hands the result to the audit session and waits.
→ The audit session returns PASS or requested changes.
→ Only then does the work session resume.

I asked whether this repeatable process could be made into a Skill. The AI kept complicating the discussion with session architecture and the limits of mechanical enforcement, so I brought in answers from other sessions and asked again.

“You really think making this a Skill is inefficient? Seriously? You’re not just making that up?”

At the same time, I kept demanding criticism rather than automatic agreement. I did not merely want to hear that my idea was right. I wanted to know whether it could truly prevent another 16-hour failure or whether I was just building myself another impressive-looking toy.

Repeated Objections Reduced the Scope of the Experiment

After several rounds of critical review, I had to admit that my simple proposal also had holes.

If the work session wrote the audit criteria itself, it could create criteria favorable to its own output. If the audit session inherited all of the builder’s reasoning and self-assessment, it might simply reread the builder’s explanation instead of reviewing independently. If dependent work started while the audit was still running, the PASS gate would mean nothing. The system might even approve an old screen after the Runtime had changed, or misremember a prior PASS after the conversation was compacted.

But I did not need to accept everything the AI proposed either. I rejected three long-lived sessions, a separate graph runtime, a database, a dashboard, an audit for every page, and full re-audits whenever a contract changed. If I built all of that, the management system would become larger than the product.

I kept only these rules:

• The main session owns both coordination and implementation.
• A separate audit session opens only for a result that actually needs a gate.
• The audit session does not receive the entire prior conversation.
• The candidate result is frozen, and neither it nor its direct dependent work changes until the audit finishes.
• An implementation defect is distinguished from a flawed direction or contract.
• If one revision and re-audit still leave a material problem, the plan is reopened instead of continuing to patch.
• When a contract changes, only affected work is rechecked.
• An auditor’s PASS never replaces my product-direction approval or deployment approval.

Most importantly, I decided not to force this structure onto every task. Attaching an audit session to a typo, a bug settled by one decisive test, or open-ended exploration without a candidate would be wasteful. I would place a Gate only where a wrong result could contaminate several downstream tasks.

Nor would one page automatically equal one node. If searching a list, opening a detail, completing an action, and returning to the original filters form one user job, that entire journey should be the review unit. On the other hand, there is no reason to audit every screen that merely repeats the same pattern.

After stripping it down this way, the simple graph I had wanted remained intact, while the reasons to stop and the evidence to review became clear.

I Moved the Minimal Graph into a Codex Skill

What I ultimately asked Codex to create was a shared Skill named custom-graph-engineering-gate. OpenAI already provided a way to save repeatable workflows as Skills and to have a main Agent call a separate subagent and collect its result.²³ I could build the round trip between a work session and an audit session inside Codex without creating a separate service.

I also told it not to inflate the implementation. We created no execution script, separate database, or graph dashboard—only the Skill instructions and the minimum metadata needed for Codex to display it.

The Skill does three main things.

First, it decides whether the current task is important enough to open a Gate. It applies only when a wrong intermediate result could damage multiple downstream results.

Second, it fixes the reviewable outcome in one sentence: the result the user should receive, the observable criteria that separate success from failure, the exact candidate being reviewed, the normal verification path, and the direct successors that PASS would unlock. The number of files changed or tests run cannot substitute for the user’s result.

Third, it restricts the transition between work and audit.

PASS              → proceed only to direct successors
CHANGES_REQUESTED → revise the same result once and re-audit
REPLAN_REQUIRED   → revisit the contract or decomposition, not merely the implementation
NOT_VERIFIABLE    → never pass without evidence

The audit session does not inherit the prior conversation. Why I was angry, how hard the work session had tried, and which parts the builder considered good are not audit evidence. The reviewer sees the original requirement and the current result directly.

If source or Runtime changes during the audit, the old verdict is discarded. If an interruption or compaction makes it impossible to reconstruct which result received which verdict, the Skill does not guess that it passed.

This is still close to the structure I proposed at the beginning. The Skill merely preserves the transition rules that are easy to forget, so the idea of “keeping one audit session” can run with the same topology in the next session too.

I Tested the Graph on a Real Contract

If I trusted the Skill merely because it had been created, I would be repeating the same mistake. I therefore handed the new Skill itself to a separate reviewer that had not inherited the earlier conversation.

First, I checked that the Skill correctly stayed out of a simple wording edit. Not using it when it was unnecessary was the first test against token waste.

Then I used an authorization-transition contract shared by several features as the first real node. I froze the candidate and sent it to the audit session. The first verdict was not PASS but CHANGES_REQUESTED. It had not defined clearly enough which point in time governed authorization, where delegated authority originated and how revocation propagated, or whether the user, grant, and target company were all bound to the same scope.

If we had expanded that contract to multiple features, we would later have had to tear apart the whole authorization boundary. The work session revised only that scope, and the same audit session reviewed it again before returning PASS. Downstream implementation and deployment remained closed the entire time.

It was a small test, but I saw the Graph Engineering effect I had wanted. The downstream edge remained closed at a node the builder had considered “good enough,” and the problem was corrected before it spread. Only after the same audit session reviewed the revised candidate and returned PASS could the next step open.

The First Experiment Passed, but It Has Not Solved the UI Failure Yet

The intended graph worked in the first test. But I cannot say that this Skill would have prevented the 16-hour UI failure. Reviewing an authorization contract with comparatively explicit source is not the same as interrupting a misguided visual direction. I have not yet tested whether it reduces total time and token use in a real large-scale UI effort. The audit session also sees the same model and Working Tree and may share the work session’s mistaken assumptions. A Skill is an instruction, not an enforcement engine.

So the next long-running task should not be judged by its number of PASS verdicts.

• Did it catch the first wrong direction before broad expansion?
• Did the rework it prevented cost more than the audit tokens?
• Did it delay the moment when I first saw a provisional result?
• Did the work and audit sessions share the same flawed contract?

If the experiment does not pay off, I should use the Gate less, not add more rules. If the same failure repeats, I should first revisit what I chose as one result and what evidence I showed the reviewer, rather than adding yet another audit stage.

Learning the term Graph Engineering did not suddenly make me good at using AI. But this time I did not hear a fashionable term and immediately install a huge framework. I kept only the graph needed for the failure I had experienced, kept challenging the AI’s oversized proposals, and turned it into a small Skill I can test again in future sessions.

So that I do not discover only after 16 hours that the entire direction was wrong.

References

1. LangChain, “3 Years of Graph Engineering with LangGraph”. An explanation of the recent term Graph Engineering and the view of designing Agent workflows as explicit graphs. ↩

2. OpenAI, “Build skills”. The official path for saving repeatable workflows as Codex Skills. ↩

3. OpenAI, “Subagents”. An explanation of how a main Agent can create separate subagents and collect their results. ↩