What IDP actually automates (and what it doesn't)

01.What IDP automates well

IDP is a pipeline. Each stage does a specific job:

• Reading. It captures documents from email, scans, PDFs and phone photos, and turns them into readable text and layout.
• Classifying. It decides whether a file is an invoice, a claim form, a contract or something else.
• Extracting. It pulls fields and tables: supplier name, invoice date, policy number, line items.
• Checking. It compares the output against rules: do the line items add up, does the date make sense, does this supplier exist in the master list?
• Posting. It writes clean data into the ERP, CRM or case system.

These are repetitive, rule-bound tasks. They are where people lose hours and make typing errors. Done well, they run in seconds and leave a log of what happened to every file.

02.What it does not automate

Some work stays with people, and that is fine.

• Judgement on unusual cases. A contract with a non-standard clause, or a claim that does not fit the usual pattern, needs someone who understands the business.
• Vendor disputes. If an invoice is wrong, the system can flag the mismatch. A person still has to talk to the supplier and agree a fix.
• Bad source data. If a form is illegible or a field was never filled in, no model can invent the answer. The fix is upstream, in how the document is created or collected.

These cases go to people through an exception queue. That is not a failure of the system. It is the design. A pipeline that pretends to handle everything will quietly post wrong data.

03.The exception lane is the product

Teams often treat the review queue as a fallback. In practice it is half the system. A good queue shows the document next to the extracted fields, says why it was flagged, lets a reviewer correct a value in a few seconds, and stores the correction.

Those corrections are valuable. They show where the system is weak, which suppliers or layouts cause trouble, and which rules are too strict or too loose. Over weeks, the exception rate should fall and the reasons should become more specific. If nobody owns the queue, it grows, and the whole project looks like it failed when it only lacked an owner.

Ask three questions about any queue you build. Who works it? How fast must they clear it? What do they do when a document is truly unreadable? If the answers are vague, fix that before go-live.

04.Setting a realistic target

A good target is 70 to 90 per cent straight through on common document types, with humans handling the rest. Set your own target after measuring a sample.

A few points on how to do that:

• Pick a sample of real documents, not the cleanest ones. Include the blurry photos and the odd layouts.
• Label the correct values by hand for each field.
• Run the pipeline and compare per field, not as one overall score.
• Decide which errors are costly. A wrong total matters more than a wrong middle initial.
• Set a target per document type, since a standard invoice and a handwritten form will not perform alike.

Straight-through rate is a useful measure, but it can be gamed. If confidence thresholds are loosened to push more documents through, errors reach your ERP. Track straight-through rate alongside the post-posting error rate, so a better number on one side cannot hide a worse number on the other.

05.A worked example

Imagine a team receiving supplier invoices by email. Most come as PDFs from a few dozen regular suppliers. Some are photos from small vendors. A few are credit notes that look like invoices.

The pipeline classifies credit notes separately, extracts header fields and line items, matches against purchase orders, and posts clean invoices. The photos from small vendors have lower confidence and go to review more often. Invoices with a total that does not match their line items are always flagged, whatever the model's confidence. Over a few weeks, the team notices that one supplier's layout causes most misreads and adds a specific handling rule for it. The exception rate drops without anyone changing the model.

That pattern, measure, find the cause, fix the rule or the layout, is what improvement looks like in practice. It is steady engineering, not a breakthrough. It also shows why a pilot on one document type teaches you more than a broad launch.

06.Where the pipeline breaks in practice

Failures cluster in a few places, and it helps to know them in advance.

Capture. A scan is rotated, cropped or photographed under a desk lamp. Multi-page files contain several documents stapled together. An email carries the invoice as a link rather than an attachment. A good intake step splits, rotates and rejects before anything expensive runs.

Tables. Headers are easy. Line items are not. A table that spans two pages, merges cells or wraps a long description onto three lines will confuse a naive extractor. Test tables separately from single fields, and check that line items add up to the stated total.

Context. A date such as 03/04/25 means different things in different countries. A supplier name may appear as a trading name on one invoice and a legal name on the next. Rules that use your master data resolve most of this. The model alone cannot.

Drift. Suppliers redesign their invoices. A pipeline that scored well in March can slip by June without any code change. Watch accuracy per supplier over time, and alert when a supplier's exception rate jumps.

Each of these has a plain fix. None of them needs a bigger model. They need testing against real files and an owner who looks at the numbers every week.

07.Common mistakes

• Judging the system on a clean demo set.
• Measuring only overall accuracy instead of per-field accuracy.
• Skipping validation rules and relying on model confidence alone.
• Having no owner for the exception queue, so it grows.
• Expecting 100 per cent straight through, then loosening checks to hit it.
• Forgetting to log which source document produced which record, which makes audits painful.

08.A note on cost and speed

Teams sometimes ask whether to use the largest model for every document. Usually not. Simple, regular documents can be read by lighter methods at lower cost, and the expensive path can be kept for hard cases. Route by document type and by confidence, and measure what each route costs per document alongside its accuracy.

09.What to do next

• List your top five document types by volume.
• Collect 100 or so real examples of each, including the messy ones.
• Write down the fields you need and what correct looks like.
• Decide who reviews exceptions and how fast.
• Pilot one document type before expanding.

IDP is not magic, and it does not remove people from the process. It moves them from typing to checking. If you are weighing where to start, Brandsmashers builds these pipelines with exception handling designed in from the first week, and a scoping call is a sensible place to test your numbers.