I came up through QA, so my first instinct with any new problem is one I have had for years: is this a candidate for automation, or does it need a person?
In testing, that question has a settled answer. You automate the regression suite because it runs the same way every time, the expected result is known, and running it by hand is a waste of a human. You do not automate exploratory testing, because the value there is a person noticing something nobody predicted. Trying to automate exploration produces a suite that passes while the product is broken.
Building with AI turns out to work the same way, and the mistake people make is the same mistake. They automate the exploratory part.
The question is not which is better
Both shapes are useful. They are answers to different questions.
An automation runs on a trigger. Something arrives, work happens, a result lands somewhere. Nobody opens anything. The human is an interrupt, pulled in only when the system cannot proceed.
A tool runs on a person. Someone opens it, works through what is in front of them, and makes decisions with the system assembling context around them.
The deciding factor is not how advanced the AI is. It is what proportion of the work is genuinely predictable, and what the cost of being wrong is.
Accounts payable is a good example because it is both
Take invoice processing. An invoice arrives, you check it against the purchase order and the receiving record, and if everything matches you pay it.
Watch someone actually do this and a pattern shows up immediately. Roughly 80 percent of invoices match cleanly and take about two minutes each. The other 20 percent are exceptions, and they consume the day. A price is off by a few dollars. The quantity billed does not match what was received. The purchase order number is missing. The vendor added a line that was never ordered.
That split is the whole design.
The 80 percent is regression testing. Known inputs, known expected result, no judgment required. Automate it.
The 20 percent is exploratory. The evidence is scattered across a purchase order, an email thread, and a receiving record. Someone has to look at it and decide. Do not automate it. Build the tool that makes it fast.
What I would build
A pipeline that runs on arrival. An invoice lands in the AP inbox and the workflow fires. It extracts the line items, matches against the purchase order and the goods receipt, applies tolerance thresholds, and either settles the invoice or classifies why it could not.
Deterministic extraction first, with a fallback. Every vendor formats invoices differently, but the fields underneath are the same. Vendor A says Net Amount, vendor B says Subtotal, and both mean the same thing. For vendors you see often, write a parser and map to a canonical schema. No model call at all. Cheaper, reproducible, auditable.
The long tail is different. You will never write a parser for the vendor who sends three invoices a year, and a brand new vendor has no samples. Those go to an LLM for extraction, a person confirms the field mapping once, and the system writes a template from that confirmation.
The nice property of that design is that it graduates. In month one most volume runs through the model. By month six most of it runs through code, and your cost per invoice has collapsed without anyone doing a migration project.
An exception queue that assembles evidence. When the pipeline cannot settle an invoice, it does not just flag it. It gathers the purchase order line, the receiving record, the price history for that item from that vendor, and any prior invoices against the same order. Then it proposes a resolution.
There is a real difference between these two outputs:
Price variance: $340
Price variance of $340. The purchase order reflects Q1 pricing. This vendor updated their price list on March 1, and the contract permits an annual adjustment. Recommend approve.
The first is detection. The second is resolution. Detection is easy and mostly unhelpful, because the person still has to go find everything.
A boundary written down in advance. Each exception type gets an explicit rule. Under a materiality threshold, resolve automatically and log the reason. Above it, route to a person with the evidence attached. Anything the system cannot classify goes to a person untouched, with what it found.
That last category matters more than it looks. A system that always has an answer is a system that is confidently wrong sometimes. One that says it does not recognize this pattern is one you can trust the rest of the time.
An audit trail on every decision. What the system saw, what it concluded, what a person decided, when, and which version of the logic produced it. In most domains that is good practice. In finance it is the requirement.
What I would not build
I would not have the model do arithmetic. Matching quantities, applying tolerances, converting currency, comparing dates. All of that is code. If you find yourself writing a prompt that asks a model to add numbers, that is a bug, not a feature. Every run re-derives the same calculation, costs tokens to do it, and introduces variance for no benefit.
I would not let it move money. The system proposes, a person approves, the ERP executes. Not because the model cannot be trusted with the logic, but because the failure mode is unrecoverable and the approval costs seconds.
I would not automate the exception path. This is the tempting one. Exceptions are where the time goes, so they look like the best target. But exceptions are exceptions precisely because they need judgment, and a system that resolves them automatically is one that quietly makes the wrong call on the cases that matter most. Make the human faster instead of removing them.
I would not build it interface first. It is easy to start with the screens because they are satisfying to build. But if the pipeline is right, most invoices never appear on a screen at all, and building the interface first tends to produce a system where everything does.
I would not demo it with clean data. Synthetic invoices hide exactly the mess that makes this hard: the scanned PDF, the missing purchase order number, the vendor who puts tax on its own line. A version that only works on tidy inputs proves nothing.
What the outcome looks like
The metric people reach for is invoices processed, and it is the wrong one. Clean invoices already took two minutes. Making them take ten seconds sounds impressive and saves almost nothing, because that was never where the day went.
The numbers that matter are the exception ones. How long does it take to resolve an exception now versus before. What share of exceptions get resolved without a person opening another system to gather context. And how much money gets caught that would otherwise have been paid wrong: the duplicate that slipped through, the price creep nobody noticed across six months of small increases, the payment terms that did not match the contract.
That last number is the one that funds the next project.
The general rule
Automate the part that is the same every time. Build a tool for the part that is different every time. Put the boundary between them somewhere explicit, write it down, and make the system say so when it does not know.
That is true for test suites and it is true for agents. The technology changed. The judgment did not.