Every October, the bug came back

In the late 1990s I worked on a migration from a mainframe to Unix. We wrote our own tests for it. One of them happened to pick a date in October, and the month printed as "010" instead of "10".

When we dug in, we found the bug wasn’t new. It had been in production for years. It showed up on the first of October and was gone on the first of November, the morning after Halloween, every year. Nobody had ever fixed it, presumably on the grounds that it always went away by itself.

The cause was small. Months were stored counting from zero, for reasons nobody could remember, so January was 0 and October was 9. To print a month, the code added a "0" in front if the stored number was below 10, then added one to show the month the way people expect. That is how October, stored as 9, ended up with an extra zero. Every other month came out right. The fix, when it finally came, was a one-line change. Finding it had taken years and a lucky date.

Every legacy system has ghosts of this kind. Some live in the code. Some live in people’s heads, like the one engineer who understands the system and is set to retire next month. Others live in the environment, in the old infrastructure and rules around a system built decades ago. In her recent article, Brownfield modernisation with AI: beyond the hype, Nearform’s Katie Roberts showed how AI agents can help modernise brownfield systems and advised starting with a map before letting agents change anything. One section of that article focuses on the hidden hazards of uncontrolled AI agents, describing how agents fill gaps with confident guesses and write tests that confirm their own assumptions.

Before work starts is the cheapest time to find out what you don’t know. Here are seven questions to answer before using AI to modernise legacy systems: four that help you draw a map, and three that check whether your tests, environment and agents are ready to use it. Answering each question closes a gap that will otherwise turn up later as an incident or a missed deadline.

1. Do you have a real map of what the system touches?

An old architecture diagram shows what was connected when someone drew it. A dependency graph generated from the code shows what the code connects today: which modules, services, databases and jobs call or read from one another. To test the map you have, pick one table the system writes and ask who else reads it. If the answer comes from memory, you may miss the other team’s reporting job that breaks when a column is renamed. The database’s query logs, or tracing in front of it, show who actually reads the table. A connection missing from the map can become a regression another team finds in production.

2. Where does the logic live outside the code?

Business rules often sit in stored procedures, in scheduled jobs defined on a server rather than in the repository, and in spreadsheets someone uploads every quarter. We’ve all opened one called "final_v3 (2).xlsx". A discount rate that a script on one machine writes to a table every night is invisible to an agent reading the repository, so list each with a location and an owner. If nobody lists a rule, the new system quietly drops it, and the business doesn’t notice until the end of next quarter. Once rules are listed, export the stored procedures, job definitions and spreadsheet macros into the repository, so the tools that read the code can read those artefacts too.

3. How far back can you see?

Some behaviour is seasonal, driven by external cycles like Black Friday traffic, or internal ones like month-end batches (or my October bug). Check how far back your logs and metrics go, because a few weeks of retention misses most of these cycles. Incident reports often show the pattern best, with similar tickets arriving at suspiciously regular intervals. The same history shows what nobody uses. A feature built eight years ago and not touched since can be retired instead of migrated, which cuts scope and cost. If retention is short, extend it now, because you can’t recover history you didn’t keep. Otherwise, the first time you see a yearly behaviour may be in the new system, after go-live.

4. Is there tribal knowledge at risk?

Treat knowledge held by any one person as a dependency with an expiry date. Version history shows the modules where a single person has made almost every change, often the ones only that person can explain, such as why the overnight job must run after the backup. If your system has that engineer, book their time now, and turn what they know into a specification they review. Once they have left, rebuilding the same knowledge from the code is slow and expensive, if it’s possible at all.

5. How will you prove that the new system behaves as intended?

The answer to this question is the evidence you need to approve the migration. Characterisation tests, which record what the system does today, are the starting point. Where you can, also replay recorded production traffic through both systems, or run them side by side, and compare the outputs. A characterisation test written for my old system would have faithfully recorded "010" as correct, so a subject matter expert, someone who knows the business, still decides which behaviours are the contract and which are bugs. Those decisions can then become property-based tests: you state a rule the output must always follow, and a tool generates inputs to break it, trying edge cases first, such as zero, the ends of a range, or the point where a number gains a digit. A model can propose these rules too, and for common knowledge it will usually get them right. Rules specific to your business still need a subject matter expert to confirm them.

6. Will the environment let your team and its agents work?

The organisations that most need modernisation often have controls designed for another era, but they apply to agents too. Check that engineers get new accounts, approved AI tools and models, a token budget sized for real use, permission to use a local database or realistic test data, and an environment where old and new versions can run side by side for the comparisons in question 5. Get a clear answer from security, too, on where the code can go and in which countries, how long it’s kept and whether it can be used for training. Non-technical setup can add weeks before the team can build anything, so start these requests as early as possible.

7. How would a newcomer, human or agent, understand the project?

An agent starts every task like a new employee on their first day, so give it what it needs to be successful: one command to build, one to run the tests, conventions enforced by automated checks, and an instructions file (usually called AGENTS.md) that agents read before starting. Keep it short, and focus on what the code can’t tell the agent, such as non-standard conventions and the traps to avoid. Procedures and reasoning that the agent can’t work out from the code (like how to run the overnight job locally or why it must run after the backup) can go in skills, which it loads only when a task calls for them, to keep the file short.

Mapping the code is the part agents speed up most. On a legacy modernisation for one of our clients, codebase mapping went from two weeks to just a few hours. The rest of the map is still human work: booking time with the people who know, and getting access approved.

Without a map, an agent explores a codebase like someone in a dark building with a flashlight, searching for text and opening files one at a time, while every file it reads costs tokens. Agents are good at drawing the part of the map that lives in the code, especially on top of a code graph built by parsing it (the dependency graph from question 1), where imports and function calls are the roads. That graph answers questions like "what calls this function?" in a few hundred tokens. The rest of the map isn’t in the code; rather, it spans the logic in stored procedures and spreadsheets, the history in logs and tickets, and the tribal knowledge in people’s heads. Humans have to draw that part and document it where agents can find it.

A March 2026 study compared an agent that queried a code graph with one that read the source files directly. The graph agent used far fewer tokens, but its answers were less accurate. My hypothesis is that the graph leaves out detail that only the source holds, so I recommend giving agents both: the code graph to navigate, like a map, and the source files to confirm, like a flashlight. Before choosing a graph tool, check that it parses every language in the estate or that you can add your own parser, because coverage usually ends where the oldest code begins.

If the specification is the contract agents answer to, a system that never had one needs a first spec. Documentation an agent writes freely from the code is hard to verify and goes stale quickly, so a more reliable approach starts with facts extracted by deterministic tools and lets agents write on top of them.

Unwind, an open source project from Nearform, starts from a deterministic inventory. It catalogues every function, class and import in the languages it supports, has agents document each layer, lists by name and line everything the documentation still misses, and flags business rules that shouldn’t be carried over. Anthropic’s code-modernization plugin is organised around business rules and approvals: it checks the environment, extracts rules that cite their source lines, asks a person to approve the plan, and independently checks that the new code behaves like the old. Both run in Claude Code, and both go as far as the rebuild. Choose Unwind when you want documentation you can check is complete, the plugin when you want rule-by-rule sign-off and a verdict on each module. The spec produced by either tool still needs reviewing by people who know the system and the business (see questions 4 and 5) before anything is built. It should be reviewed more closely than the code itself, because every later step answers to it.

BMAD, an open source framework in which a written specification drives what agents build, can then carry the work forward. BMAD deliberately keeps its context small and has its build step read the code directly, which works when all the rules live in the code. On a legacy system, many of them don’t (see question 2), so give BMAD the map and treat the analysis as the requirements its product owner agent slices into work. Its build step then implements each piece for an engineer to review. Ask the agent for changes instead of editing its output by hand, so its checks run and the documentation stays current. BMAD now supports unattended runs, but don’t count on them for code this old.

Picture that October bug in a migration today, with the map from this post. The history shows tickets arriving at the same time every year, a hint that something keeps coming back. The call graph turns that hint into a place, leading from the report with the wrong month to the function that formats it. An agent then writes a property-based test for that function. With only twelve months to try, the tool covers every one, so the month where the number gains a digit can’t slip through.

Generating inputs is the easy part. The test also needs to know what a correct result looks like, and a characterisation test would have taken that from the old system and passed. Here a small model could supply the rule, because what a month looks like is common knowledge. For most of what a legacy system gets wrong, it takes someone who knows the business. Tools show where a ghost can hide. People recognise it when it appears.

Ours came back every October. Yours might come back at any moment, so expect ghosts. Start with the questions that take the longest to answer, because they set your earliest start date: book time with the people who hold the knowledge, and send the access requests this week. If you’d like a partner in drawing the map for your own system, talk to Nearform.