From legacy PHP to verifiable parity: a DFD migration case

In the first article in this series I argued that with agents, development cost does not disappear: it moves upstream to specification and downstream to verification. In the third, experimentation revealed that the most important signal came from a lying gate rather than the tests: a CI pipeline that could not fail.

This fourth test covers different ground: a migration, rather than a new feature. A legacy PHP management application still in use, and a Laravel conversion that must reach functional parity before switchover. Another detail changes the significance: unlike earlier experiments with Claude Code, the technical work was led by Codex, OpenAI’s development agent, using DFD, DoD-First Driven, a framework I am developing, and its CLI.

This too is an experiment, not a demonstration. One project, real and imperfect material, and some checks I could not rerun because of session limitations. What follows is an honest case, with its gaps declared.

Why migration is the perfect test of the argument

A new feature gives you room: if the spec is vague, the “right” behaviour is partly a choice. A migration does not. The right behaviour already exists in the live system. The job is not invention; it is demonstrating equivalence. Cost moves entirely to specifying what must remain unchanged and verifying that it does. Exactly the first article’s argument, without escape routes.

The old application’s rules were mostly implicit, living in PHP code and users’ habits rather than documentation. DFD helped turn them into explicit, verifiable decisions.

Implicit legacy rule	What DFD made it
Roles and permissions	Explicit decision with test cases
Annual fees	Specification plus checks against historical amounts
Dinner separate from membership fee	Its own testable rule
Numbers assigned at delivery	Decision on when and how assignment happens
Seniority	Explicit, tested criterion
Paper retention of signatures	Declared process constraint

The centrepiece was the parity matrix: a table stating item by item whether the new system is equivalent, different or still awaiting acceptance testing. It is the migration version of the first article’s Definition of Done: instead of “what must the feature do?”, “what must remain identical, and how do I prove it?”.

What worked

Here the argument holds, visibly.

• Automated checks covered application parity. Rather than “it looks the same”, a test fails when it stops being equivalent.
• The new database remained separate from the legacy system. No writes to the live system: the simplest safety net, and one too often skipped.
• Import preserved historical amounts and avoided automatically merging ambiguous records, the worst import temptation: “cleaning” data by making it disappear.
• New accounts used bcrypt password hashes. Bcrypt is deliberately slow to resist brute-force attacks. Authentication and data migration were mandatory-review areas in the first article; here they were treated accordingly.
• No CI: the project was in development-only mode, a stage DFD explicitly supports. CI was premature, but local checks were recorded as evidence.

That last point also exposes the crack.

Where real material pushed back

The most instructive obstacles came from real data rather than the agent. This is the first article’s “requirements emerge through use”, in an archaeological form.

• The dump did not provide a reliable annual price list: historical rates needed reconstruction.
• It contained duplicate membership numbers.
• It had an “online” status the old code did not even interpret.
• There was no initial fixture dataset, no known baseline for repeatable tests.

Three process problems added friction: DFD gates were not always aligned with real activities, Git SSH access did not work in that session, and the local runtime prevented rerunning tests.

The right response to such data is not forcing it to look clean. It is preservation and traceability:

Data issue	Correction
No reliable annual price list	Inactive historical rates reconstructed from actual membership amounts
Duplicate numbers	Conflicting numbers retained as provenance, without reassigning them as active
Uninterpreted “online” status	Status preserved, not reinterpreted
Legacy administrative users	Excluded from import

This is the first article’s lesson from another angle. There the risk was code nobody requested; here it is reinterpretation nobody requested. In a migration, inventing meaning is a bug discovered months later.

I transferred the changes to GitHub through a dedicated branch and PR, checking files before merging and excluding dumps and local artefacts. No blind copying.

The signal, again: a gate saying “done”

Here came the surprise, despite having seen it in the third article with another tool and another agent.

DFD state updates did not always match real work. The result was stale gates. Matrix cells said “tested” while the runtime would not even allow tests to be rerun; entries said “ready” based on a merged PR rather than an observed acceptance test.

It is exactly the same failure as the CI that could not fail, one level higher:

Third article	This case
Green CI because of cmd | tee file	Green DFD gate because state had not been updated
The seal did not prove what it claimed	The matrix did not prove the parity it claimed
The real signal was in the pipeline, not the tests	The real signal was the gap between gate and reality

This is no more about DFD than the earlier case was about GitHub Actions. DFD has tools to prevent it: it distinguishes development-only from release preparation, and a merged PR in the former does not justify saying “ready”. The gap was in the discipline of keeping state updated, rather than the framework. Which returns us to the real question: what does a green gate prove? A merged pull request proves code was integrated. Alone, it does not prove operational acceptance, successful restoration or release readiness. A gate making those claims must connect to an observed fact, such as a rerun test or an executed restore, rather than state somebody forgot to update.

The link to production validation is direct: shift-right works because production signals are facts, rather than declarations. A gate disconnected from reality is the exact opposite of a canary.

What I take away

DFD helps when it records decisions, evidence and uncertainty alongside code. It makes the first article’s “living artefact” concrete. The parity matrix let us track what was equivalent, different or awaiting testing, while uncertainties stayed visible rather than being quietly resolved. In a migration where correct behaviour already exists, it moves cost precisely where it belongs.

But a gate tool is only as valuable as its fidelity to reality. Here the series takes an uncomfortable turn: having the right gates is not enough; they must tell the truth when you look at them. A green status disconnected from completed work is worse than no gate, because it gives permission to trust.

Two experiments, two tools, two different agents, the same discovery. I am starting to think this is no coincidence: when cost moves to verification, the weakest point becomes the mechanism that declares “verified”.

Postscript. In the next case, back with Claude Code and two Taita features taken to production, the gate told the truth, but only on the machine where I had tested: evidence logs were not reaching the repository. Read From external feedback to release.

Declared limitations

• One real project with imperfect data; no replications.
• Checks could not be rerun in the session because of local runtime and SSH limitations: some evidence was collected earlier, rather than reproducible on demand.
• Historical rates reconstructed from actual amounts are estimates, not original source data.
• The experiment used Codex and DFD. There is no controlled comparison with another agent or framework on the same project.