[🛠] AI Orchestration #5: Redesigning the Coordinator–Worker Split I Had Abandoned

✨ GPT-5.6 Sol’s Summary

A record of abandoning distributed work when worker messages made the coordinator lose context, then redesigning it around file reports and coordinator-only commits after running into the bottleneck of a single session.

I Once Gave Up Separating the Coordinator and Workers

Last month, I connected a coordinator and workers in a real workflow. One goal was divided into several tasks, and the coordinator reviewed and integrated what the workers implemented.

The division of work itself was fast. Independent features could be implemented at the same time, while the coordinator focused only on the product’s overall direction and integration. The problem was how workers delivered their results.

Whenever workers finished or got blocked, they sent a report into the coordinator session. At first this seemed natural. As the number of workers grew, however, reports kept piling up in the coordinator conversation, and every incoming message stopped the current work and pulled it toward another task.

I would be reading one worker’s test result, switch to another worker’s Git index conflict, and then receive yet another completion report just as I tried to return to the original implementation. The conversation grew rapidly. As compaction repeated, more time went into reconstructing the goal, deployment prohibitions, and current staged state. At some point, the coordinator looked less like an integrator and more like an inbox for worker messages.

I eventually abandoned the coordinator–worker split itself. I even deleted the coordination Skill I had built on impulse. I thought doing everything in one main session would at least keep the context from being dragged in several directions.

Returning to One Session Revealed Why Distributed Work Was Necessary

Once I continued large product work in one session, the opposite bottleneck became obvious.

One session had to handle implementation, tests, documentation alignment, checks on physical devices, and Runtime preparation in sequence. If a priority question or a handoff from another session arrived midway, I first had to reconstruct the state of the work already in progress. Finishing one thing thoroughly made the whole effort longer, while handling an urgent item first required another judgment about how far the existing task had to be closed out.

Distributed work was not the problem. Holding even independent tasks inside one session was the inefficient part. Separating the coordinator and workers was still the most efficient approach. Reviving the same structure in which workers entered the coordinator’s current conversation directly would only repeat the earlier failure.

This time, instead of giving up distributed work, I decided to remove the delivery path that interrupted the coordinator’s context.

Even a One-Line Notification Opened the Same Door

I first wondered whether workers could send a short message instead of a long report.

READY task-042: report path

But length was not the problem. The instant a message entered the coordinator conversation, Codex could easily treat it as newer context than the work already in progress. Even one line left the same door open for switching tasks.

So workers no longer send messages to the coordinator. Completion, blocked, and failure states are all written only to report files. The coordinator checks the inbox only after finishing its own work at a safe checkpoint or commit.

The arrival of a report is not an event that interrupts current work. It merely adds one file to the next review queue.

Worktrees Were Clean but Drifted Away from the Latest State

Next, I considered giving each worker a separate Worktree and Branch. The Git index and uncommitted changes would never mix, so it looked like the cleanest option.

In a real project, however, a larger problem appeared. Several features kept touching the same API, schema, and screen. Improvements completed in one session were not immediately visible in the other Worktrees. Everyone worked cleanly, but integration revealed a growing stack of changes built against outdated structures. That was when the amount of rework became ugly.

This time I chose the opposite structure.

One main Working Tree
├─ Multiple workers see the latest changes together
├─ Workers edit only assigned files and hunks
├─ Workers do not stage or commit
└─ Only the coordinator reviews, stages, and commits

A shared Working Tree would tangle immediately without limits. By default, one file has one writer. Parallel edits to the same file are allowed only when the hunks are clearly separated. Shared schemas, import cleanup, formatters, and other changes that move surrounding code are serialized.

Worker tests are not accepted as final evidence either. A worker may have run them while another worker’s uncommitted changes were present. Workers run checks for their own scope; the coordinator freezes the change set and runs the full regression and build once.

Reports Became Review Contracts Instead of Conversation Messages

I rebuilt the deleted coordination Skill around these principles. The new custom-coordinate-parallel-workers Skill contains this flow.

The coordinator assigns a task id, desired user outcome, and writable paths. The worker completes implementation, scoped verification, and temporary-process cleanup, then creates a report atomically and stops. The report lists the files actually changed, check results, overlaps with other work, unfinished scope, and a proposed commit.

The coordinator does not reread every conversation. It first scans the inbox for the task id, status, title, and check summary, then chooses one task to integrate next. After confirming that the report matches the actual diff, it stages and commits only the exact hunks.

Once a release candidate is chosen, a freeze prevents further worker mutations. Instead of repeating full tests while changes continue to arrive, current-state documentation, full regressions, builds, and deployment verification proceed only after the release set stops moving.

The report is no longer a simple work log. It has become a review contract that lets the coordinator decide whether the change can move into the next state.

Now It Has to Run in Real Work

I tested the Skill’s state-management tools in an isolated temporary environment. Normal completion, writable-path conflicts, rework after a change request, freeze rejection while work was running, and deferral to the next batch all passed.

I have not yet operated several real Codex sessions this way. The next parallel task must show whether file-only reports truly disturb the coordinator’s context less and whether path ownership in a shared Working Tree is respected as well as expected.

The conclusion resembles where I started, but the reason is now clearer. The coordinator and workers need to be separated. What failed was not distributed work; it was pushing every worker report into the coordinator’s current conversation in real time.

The change I expect is not simply to launch more workers. It is to let the coordinator keep its own context until its work is finished. The speed of parallel work depended less on the number of workers than on how rarely the person integrating their results was pulled away.