TL;DR: Getting an AI agent to do the task is often the fast part. Production starts when its output has to survive the real work around it: sector rules, team templates, source hierarchies, controls, exceptions and sign-off.
A credit analyst on a commercial lending team watched the valuation agent work through a stack of property reports and draft the memo he would normally prepare for sign-off. The agent took a few minutes. Allowing time to read the draft properly and check it against the reports, the whole thing could be done in around ten minutes, compared with an hour or more doing it manually.
The useful part came next. He started pointing out what the agent did not yet know: the kinds of property he actually underwrites, the sector differences that change how a valuation should be read, and the context he would need before he could rely on the output.
We already knew the agent could do the task. What mattered now was everything the credit team knew about doing that task properly.
Some of that was written down. Some lived in templates, review processes and existing data. Some sat with people who had been doing the work for years and had never needed to explain why one case should be treated differently from another.
The requirements were already in the work
“Make it know my sector” was not a one-off. It was the most common note once an agent could do the task at all.
The valuation agent did not need to become a shelf of separate valuation agents. It needed to know that a petrol station, a warehouse and an office block are three different valuation problems wearing the same word. Each sector reads value differently, so the reasoning has to change with the property in front of it.
The same thing happens elsewhere. A credit paper has to recognise the kind of borrower in front of it and produce the template the committee expects. Reconciliation has to know which two documents it is holding because what counts as a discrepancy depends entirely on the pair.
The work was not building more agents. It was getting the system to recognise what had actually arrived.
At another bank, I scoped a review process assuming they ran three forms - one for commercial property, one for residential and one for land. They did not. They ran one, and the property type determined which parts applied. The routing I was about to build in triplicate was already sitting in their own template.
I keep finding versions of this. The logic we need is often already there, in a template, a control, an existing process or simply in the way an experienced person reads a file. The difficult part is finding it and making it explicit enough for the system to use, without turning the user into the person who has to navigate all of that complexity.
Where we can infer the route from the work itself, we do. The task tells us the workflow, the documents tell us something about the borrower and sector, and existing data can supply the business line or template.
The person using the agent should not have to understand the system diagram before they can get on with the job.
What was still missing
A general assistant can get surprisingly far and still stop short of what a bank needs in production.
One person we worked with had already built his own version of a recurring review using a general assistant. He did not need us to prove that the model could do the task. He had already done that.
The questions were no longer really about the model. They were about the work. Which source wins when two disagree? What needs to happen the same way every time? Where does variation matter? What still needs a person?
That is not tuning. It is the institution’s own knowledge becoming system design.
I now ask a second question as soon as a prototype works: what has to surround this answer before it can travel through the business?
That question changes the way apparently simple use cases look. Reading bank statements for the transactions that actually move a lending decision, or turning accounts into the comparison table a credit team already uses, can look like summarisation from a distance. The model can certainly summarise them. The useful part is knowing what matters in this decision, what takes precedence and what the next person in the process expects to see.
Banks already encode a lot of that knowledge in policies, templates, controls, processes and delegated authorities. AI does not invent the operating logic. It gives us another place where that logic has to become explicit enough for a system to act on it.
This is the work after it works
Most of this happens after the prototype has already proved the point. A new sector brings exceptions nobody thought to write down because experienced people had always handled them without discussion. A source hierarchy that sounded obvious in a workshop becomes less obvious when two records disagree. A template that looked standard turns out not to be.
Sometimes the model is helpful in exactly the wrong way. On one build, the product matcher keyed off names, and one product’s name sat inside another as a substring, so a whole category of loans was being checked against limits that belonged to something else.
Two test cases should have exposed it, but the final answers still came back correct. The model appears to have picked up from the surrounding context that the limit did not fit the case and produced the right answer anyway. Looking only at the output, there was nothing obvious to tell us the matcher underneath had failed.
We found it by measuring the matcher on its own rather than relying on the final answer as the signal. That measurement is now a test that runs every time. It changed how I think about production quality - end-to-end correctness matters, but so does knowing whether the components underneath are behaving as intended.
A lot of the work after a prototype looks like this. Something an experienced person knows becomes a rule. An awkward case becomes a test. A distinction buried in a template becomes routing. And as you build, you keep making decisions about what the system can own and what should stay with the person accountable for the outcome.
Getting an agent to work can be fast. Production is teaching it how the institution actually works.
Questions I get asked
What happens after an AI agent starts working?
Production begins. The focus shifts from whether the model can do the task to whether the surrounding behaviour is dependable: which template applies, which sources take precedence, how the system recognises the case, which checks run, how exceptions are handled and which decisions remain with a person.
Why is a capable general model not enough for banking work?
Because it does not arrive with the bank’s own operating logic. That sits in templates, source hierarchies, sector knowledge, controls, exceptions and review processes, and has to be made explicit enough for the system to use reliably.
What is the difference between a strong demo and real adoption?
A demo shows capability. Adoption depends on whether the system fits the work well enough that people can trace the evidence, understand where judgement sits and rely on the output inside a live decision.
I write about AI adoption in commercial and corporate banking, and what actually happens when agents go into production.
