Once AI starts drafting customer replies or reading supplier invoices, someone asks the obvious question: does a person need to check all of this?
Get the answer wrong and you lose either way. If nobody checks, mistakes reach customers and the books. If someone checks everything, you save little time. Worse, after a few weeks of clicking “approve” on items that are almost always right, the reviewer stops really looking. You end up paying for a check that no longer catches anything.
“Human in the loop” means deciding where a person sits in an AI process, what they look at and what they’re responsible for. This post covers five patterns, how to choose between them, how to design the review step so it’s quick, and how to move the line as you learn what the system gets right.
Five ways to keep a person in the process
1. Review every output
AI drafts, a person reads everything before anything happens. This is right for the first weeks of any new process, for low volumes, and for outputs that go out under someone’s name, such as a reply to a complaint or a letter from a law firm. You save the time spent typing and searching, but not the reading time.
2. Review by exception
AI handles routine cases on its own, and a person only sees the items that something flagged. Flags come from two places. Rules you write: the amount is over a limit, the supplier is new, the VAT doesn’t add up. And the system itself: a field it couldn’t read, or an answer it couldn’t find in the source.
This is where most of the time saving comes from. It only works if the flags catch the problems, and you only know that after measuring. Prefer checks you can verify over a model’s own sense of how sure it is. “The totals add up and the order number exists” is a stronger signal than “the model says it’s 95% confident”.
3. Approval before anything irreversible
AI prepares, a person presses the button. Use this for payments, deleting data, sending a contract, rejecting a candidate or refunding above a limit. The AI does most of the work, and the person makes one decision with everything they need on one screen.
4. Sampling audits
After the fact, a person checks a random sample, for example 20 of last week’s automatically processed invoices. It catches drift: a supplier changes its invoice layout, the provider updates the model, customers start asking a new kind of question. It’s cheap, and it’s the only way to know your exception rules still work.
5. Escalation
The AI recognises that it shouldn’t handle something and hands it to a person with context: an angry customer, a legal threat, a request outside its scope. The hand-over should include a short summary, so the person doesn’t start from scratch and the customer doesn’t have to repeat themselves.
Most real processes combine several of these. Here is a common setup for supplier invoices:
Review by exception, step by step
- Invoice arrives By email or upload
- AI extracts fields Supplier, amounts, VAT, order number
- Rules check Totals, order match, known bank account
- Person reviews exceptions Only invoices that failed a check
- Posted to accounting
You can think of the patterns as points on one line, from a person doing the work with AI help to AI acting alone:
How much the person checks
- AI suggests Person writes, AI offers a draft 1
- Person checks everything Every output reviewed first 2
- Person checks exceptions Rules decide what to review 3
- Person audits samples Review after the fact 4
- No person Only for low-stakes, reversible tasks 5
← Person does the work AI acts alone →
How to decide where the person sits
Decide per step, not per process. In one invoice process, reading the fields can run with sampling while paying the invoice always needs approval. Four questions help:
- What does a mistake cost, and who pays for it? A wrong product category in the CRM costs nothing. A wrong bank account costs the whole payment.
- Can it be undone? A draft can be deleted. A sent email can’t, and a payment rarely can.
- Can a rule check it? If the totals must add up, a rule can check every item and a person is only needed when the rule fails. If quality is a matter of judgement, such as the tone of a reply to a complaint, you need a person or a careful sample.
- Does the law require a person? Under GDPR, people have the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects on them (Article 22). Hiring, credit and access to services are the usual examples. The EU AI Act also requires human oversight for high-risk AI systems, which include AI used to recruit and manage staff. We cover both in AI, GDPR and the EU AI Act.
Plotting your steps by volume and the cost of a mistake gives a quick first answer:
A first guess at the review pattern
- 1 Tagging support tickets
- 2 Supplier invoices
- 3 Order-status replies
- 4 Replies to complaints
- 5 Rejecting a candidate
- 6 Payments
Treat the matrix as a starting point. Payments sit high on volume, but because they can’t be undone, they still need approval for every batch.
Four examples
Supplier invoices
Take a distributor that receives several hundred supplier invoices a month. For the first four weeks, a person reviews every extraction and every correction is recorded. After that, invoices from known suppliers that match an order and pass all checks are posted automatically. New suppliers, anything over a set amount and any change of bank details go to a person. Changed bank details always go to a person, because that is the classic invoice fraud pattern. More on this in AI document processing.
Customer replies
A support team starts with suggested replies. AI drafts, the agent edits and sends, and the system records how much the agent changed. For narrow request types where drafts are almost always sent unchanged, such as order status or opening hours, replies can later go out automatically, with a weekly sample. Complaints, refunds above a limit and anything that sounds like a legal threat stay with a person.
Hiring
In a recruitment agency, AI can turn CVs into structured fields, draft job ads and summarise interview notes. The decision about who moves forward and who is rejected stays with a person. That person should see the actual CV, not just a score, and should be able to disagree with it. GDPR and the AI Act both point the same way here. See AI in HR and recruitment for the details.
Payments
No autonomous payments. AI can match incoming payments to invoices and prepare an outgoing payment batch. A person approves the batch, and above a set amount a second person does too. Fixed limits beat any AI judgement here.
Design the review step so it’s fast
Most of the cost of human in the loop is the time people spend on the review screen. That screen deserves as much design as the AI step.
- Show the source next to the result. The invoice PDF beside the extracted fields. The customer’s email above the draft reply.
- Highlight what to look at. Mark the fields that failed a check or that the system wasn’t sure about. Nobody should read 30 fields to find the two that matter.
- Say why it was flagged. “Bank account differs from the last invoice” is more useful than a red dot.
- One action per decision. Approve, edit or reject, each with one click or one key.
- Ask for a reason when someone edits or rejects. A short pick-list is enough. Those reasons tell you what to fix next.
- Keep the queue short and sorted by urgency. A queue of 400 items gets skimmed, not reviewed.
Reviewing an invoice
Without a review screen
- PDF open in one window
- Accounting system in another
- Every field compared by eye
- Odd cases discussed by email
With a good review screen
- PDF and fields side by side
- Only flagged fields highlighted, with the reason
- Approve or fix with one click
- Reason for every correction recorded
Watch for rubber-stamping
When nearly every item is right, people stop reading. This tendency to over-rely on an automated system has a name, automation bias, and the EU AI Act mentions it explicitly in its rules on human oversight. Some ways to counter it:
- Track how long reviews take. Two seconds per invoice means nobody is reading.
- Track the correction rate. Zero corrections for weeks is suspicious, not reassuring.
- Plant a known mistake now and then. If it isn’t caught, your review isn’t working.
- Rotate reviewers and keep queues small enough to read properly.
Move the line as trust grows
Start with full review. Record every correction by type. When a category of items has had almost no corrections for several weeks at real volume, move it to review by exception. Keep the sampling audit running.
Move the line back when something changes: a new model version, a new type of supplier or request, a rise in complaints. Write the criteria down before you start, for example “three weeks with fewer than one correction per hundred items”. Then moving the line is a decision someone made, not something that happened because people got busy. For how far to take this, see when to use AI agents.
Someone has to own the outcome
“The AI did it” isn’t an answer a client, an auditor or a regulator accepts. Each process needs a named owner who sets the review rules and looks at the numbers every week. The system should log what the AI proposed, what the person changed, who approved it and when.
That kind of record is useful well beyond AI. In the platform we built for Dilly Labs, financial institutions keep a record of each vendor decision and the reasons behind it, so they can answer questions from colleagues, the board or regulators later.
Checklist
Before a process goes live
- A named owner for the process
- Review rules written down What is always checked, what by exception, what by sample
- Approval for anything irreversible
- A review screen built for speed Source beside result, flagged fields highlighted
- Every correction logged with a reason
- A weekly sample audit Even when everything looks fine
- Criteria for moving the line, agreed in advance
- A way for the AI to hand over to a person
The review step is usually where an AI process succeeds or fails in daily use, and it’s a normal part of what we build. If you have a process in mind, tell us how it runs today.
