Writing a migration plan for an agent to run

byStefan JohanssononOct 21, 2026read10 mininsoftware

The Aardink move to Kotlin Multiplatform was planned in a single markdown file, KMP_MIGRATION_PLAN.md, and the plan was written for an agent to carry out. Its own status line says so: “written to be executed directly by an implementation agent: every section gives concrete file paths, Gradle snippets, and Kotlin signatures rather than general guidance.”

The series covers what the migration did. This post is about the plan, how the agent ran it, what a review found after five PRs, and what I would put in the next plan from day one.

What went into the plan

The plan is about 1,500 lines. It replaced an earlier sketch that I threw out. The sketch pinned stale versions, assumed an Android plugin that does not work with AGP 9.4, and proposed rewriting the LSP transport as a Flow API, which would have broken binary compatibility for every Android user. None of that was wrong as an idea. It was wrong for this repo, and a plan that is wrong for the repo is worse than no plan, because the agent will follow it.

The second version opened with a decisions table. Four rows: which platforms, how to migrate text input, where the Aardflex grammar lives, and how the web build ships. Each row has a reason. The reasons are there so that whoever reads the plan later, agent or human, does not reopen a decision I had already made.

After that came the target module layout, the Gradle configuration with real snippets, the portability fixes needed in commonMain, and twelve numbered PRs. Each PR had a definition of done. PR 3, converting :editor to multiplatform, was done when the JVM and wasm tests were green, the screenshots were unchanged, the consumer smoke app still built, and the first ABI dump was committed.

A risk register at the end listed what I was not sure about and what to do if it went wrong. Whether the ABI task names were right. Whether @JsExport in a library survived into a consuming executable. Whether the scrollbar API existed on wasm.

I reckon this is the right shape. Decisions with reasons, concrete steps, a check per step, and a list of known unknowns. The agent ran PRs 0 through 5 in one sitting on 16 September 2026. The code compiled for Android, JVM and wasm, no tests were dropped, and the Android artefact still resolved. On the surface it had gone well.

What the review found after five PRs

Before PR 6 I ran a review. Three audits in parallel: one on :editor, one on :languages and :languages-lsp, and one on build, CI and docs. Each compared the merged state against the plan’s own definitions of done. The worst findings were then checked again by hand in the source, because a reviewer that hallucinates a bug is as much use as an implementer that hallucinates a fix.

The headline was eight items the PR descriptions claimed as done that were not there.

WebSocketLspTransport, the browser transport for the LSP client, was an empty, untracked directory. Two other files referred to the class by name. The PR 5 commit message never mentioned it.
PlatformInfo, the expect/actual that tells the editor whether it is on a Mac, did not exist. The keyboard shortcut work in PR 6 depended on it.
EditorTypography did not exist. Nine call sites still hardcoded FontFamily.Monospace.
Dispatcher injection was half done. The new EditorDispatchers object existed but nothing in :editor used it. Seven withContext(Dispatchers.Default) calls were still in the layout code.
The first ABI dump, the gate that was meant to prove every PR after 3 was purely additive, had never been taken. No abiValidation block, no .api files, no tasks.
The Dokka convention plugin still hooked the old Android plugin, which no longer fired, so all three libraries published an empty javadoc jar.
The tryLock fallback path in the rewritten LspClient had zero test coverage. The test file was a one-to-one rename of the old 22 tests.
The tests PR 1 owed for the TextFieldState migration were not written.

Worse than the missing items: CI on main had been red since PR 3. The workflow ran :editor:test and :editor:lint, and neither task exists once a module is on the multiplatform Android plugin. PRs 4 and 5 merged with nothing checking them. The release workflow had the same six dead tasks, so a release tag would have failed on the spot. The screenshot verify step was not in CI at all, which means the “zero pixel diffs” gate was resting on the agent saying so in a commit message.

The review also found two real concurrency regressions in the LspClient rewrite, where a Mutex replaced @Synchronized. Under cancellation the receive loop skipped its teardown, so the transport never closed. And listener add and remove had become unordered, so a diagnostics listener could be registered after the notification it was waiting for had already been dropped. The part 2 post covers those in detail.

One more that I find funny in hindsight. The plan’s own sketch of WebSocketLspTransport had three bugs in it. A double-resumed continuation, no close on cancellation, and handlers that were set up and then never replaced. The review’s advice was not to copy it. The agent had not got that far, so no harm done, but the plan had been ready to hand it a bug.

Why the plan did not catch it

The plan had a definition of done per PR. The agent read it, did most of the work, and wrote a commit message that said the PR was done. Nothing between the commit message and main disagreed with it.

That is the whole failure. The definition of done was prose, and prose is checked by whoever reads it. When the reader is the same agent that wrote the code, and the CI that was meant to be the second reader is silently broken, “done” means “I believe I am done”. Eight times, that belief was wrong.

The pattern in the eight items is the same every time. The work needed a new file, the agent did the surrounding work, and the new file never got created. Each miss was small. None of them stopped the build. So they did not get noticed until a reviewer sat down with the plan in one hand and the tree in the other.

What changed after

PR 5.5 went in before PR 6. The review’s remediation section became the plan for it, ordered so that each step unblocked the next. CI and scripts first, so there was a working gate. Then the ABI baseline and Dokka. Then the LspClient fixes. Then the missing files. Then the owed tests. Then the docs.

Three things changed in how the plan itself is written.

“As implemented” sections. The agent had made several deliberate deviations from the plan that were correct. Compose plugins on modules with no Compose code, because the wasm test runner needs the Skiko runtime. binaries.executable() on library targets because of a Compose Multiplatform bug. No convention plugin at all, because buildSrc and versioned plugin aliases fight over the classpath. All justified, all undocumented. The next agent to read the plan would have “fixed” them back. Every section that was implemented differently now has an “as implemented” block saying what differs and why, with the line “Do not fix them back to what the section above sketches.”

Gates a machine checks. Every claim that used to live in a commit message now has a task that fails. The ABI dump is committed and checkKotlinAbi runs in CI, so an API removal fails the build. The consumer smoke script asserts that aardink-android is in the resolved dependency tree, instead of printing the tree for me to squint at. Roborazzi runs in verify mode in CI. For 0.6 there is a performance gate: a script that types into the web editor, measures keystroke to idle at p50 and p95, counts dropped keys, and fails the build when a number is over its target.

A progress table at the top. The plan now opens with a table of PR, state, and a note on where the implementation deviated. It is the first thing an agent reads and it points at the “as implemented” sections, the review, and the successor plan for 0.6. The plan is a historical record now, and the table says so.

What I would put in the next plan from day one

The honest answer is that the plan was fine and the gates were not. So the next plan starts with the gates.

PR 0 is the checks, and the checks must be able to fail. The migration did have a PR 0 that added screenshots and the consumer smoke app. What it did not do was prove each check could go red. The next plan has a step for it: break the thing on purpose, watch the gate fail, put it back.
Every definition of done names a command, not a sentence. “ABI dump committed” becomes “./gradlew checkKotlinAbi passes and editor/api/ is non-empty”. An agent can run a command. It cannot run a sentence.
Every new file the plan says to create is listed by path in the definition of done. The eight misses were all files that did not get created. A list of paths is cheap to write and cheap to check with ls.
CI is in the definition of done for the PR that changes the build. PR 3 changed what tasks existed. Its definition of done should have been “CI green on this PR”, and it would have caught the dead task names before PRs 4 and 5 merged on top.
Review against the plan before the next phase, not at the end. The review after PR 5 was cheap next to what it saved. It kept the rest of the migration off a broken base. Reviewing after PR 10 would have found the same eight items under five more PRs of code that depended on them.
Sketches in the plan are marked as sketches. The WebSocketLspTransport code in the plan was illustrative. The plan did not say so. Now it does.

The hard work is up front

I am wary of taking a general lesson from one migration. But I keep landing on the same thing, so here it is.

The agent did not fail at Kotlin. It did not fail at Gradle, or coroutines, or the Android plugin’s quirks, and when it hit a real blocker like the buildSrc classpath fight, it worked out a correct answer on its own. Language, syntax, framework knowledge, architecture of the mechanical kind: all of that can be handed over now and the result is good.

What it needed from me was knowing what done looks like, and writing that down in a form that can be checked. Every problem in the review traces back to a place where I had written a sentence instead of a check, or an assumption instead of a decision. Where the plan was precise, the agent was precise. Where the plan was loose, the agent filled the gap with confidence, which is exactly what you would expect from a tool that is only as good as its prompt.

That shifts where the engineering effort goes. The hard part of this migration was not the twelve PRs. It was the time spent writing the plan properly, the time spent reviewing against it, and the discipline to put a machine-checked gate in front of every claim. That work used to be a nice-to-have, something a well-run team did when they had time. With an agent doing the implementation it is the job. Knowing what to ask for, in enough detail that the answer can be verified, is the skill that is now hard to delegate. The rest, more and more, is not.

So do the hard work up front. Decide what you want and why. Write it down so that a machine can check it. Then let the agent have a crack, and check.