Undisclosed Industrial Distributor
Industrial Distribution / AI Implementation / Document Automation / Agent Platforms / ERP Integration / AWS
Executive brief
68% of Purchase Orders Now Enter the ERP Untouched
A regional industrial distributor received roughly 900 purchase orders a week as email bodies, PDF attachments, and scans, all rekeyed by customer service representatives into the ERP. We built and shipped an agent that extracts order lines, resolves free-text part descriptions against an 18,000-SKU product master, validates pricing and availability, and writes clean orders back through the ERP’s API. Orders the system is not confident about route to a review console with the extracted evidence attached. The first production cutover happened in week 9 at 31% straight-through; by week 14 it was 68%, and the client’s platform team owned the runbook.
Orders entered without human touch
0% → 68%
week 14
Median order-entry handling time
9m 40s → 2m 10s
measured at week 14
Order-entry error rate
2.7% → 0.6%
rolling 30 days
Delivered a production order-entry agent in 14 weeks that now processes 68% of inbound purchase orders end to end without a human touch.
Cut median order-entry handling time from 9m 40s to 2m 10s, and reduced wrong-SKU and wrong-quantity errors from 2.7% to 0.6%.
Built a confidence-routed review console so low-confidence orders reach a human with the evidence attached, rather than being silently guessed at.
Stood up an evaluation harness over 600 labelled historical orders, so prompt, model, and catalogue changes are regression-tested before release.
Engagement shape
Duration
14 wks
Phases
5
Deliverables
7
Orders entered without human touch
0% → 68%
How it ran
Evaluation-first delivery with a conservative production cutover at week 9 and staged threshold increases thereafter
A 14-week implementation put an order-entry agent into production that reads inbound purchase orders, validates them against the ERP, and writes 68% of them straight through.
Starting position
What Undisclosed Industrial Distributor was working against.
Customer service representatives were rekeying roughly 900 purchase orders a week. Each order took nine to ten minutes, most of it spent matching free-text part descriptions against the product master and chasing pricing exceptions. Order-entry errors ran at 2.7%, and each one generated a return, a credit, or a lost customer. An off-the-shelf OCR product had been trialled two years earlier and abandoned because it handled the clean orders — which were never the problem — and failed on everything else.
Constraints we worked inside
6 in playEvery one of these ruled an easier option out. They are the reason the approach looks the way it does.
Order documents in nine recognizably different formats plus a long tail of one-off customer templates, including scans of printouts.
An 18,000-SKU product master with inconsistent naming, historical aliases, and customer-specific part numbers.
A pricing model with customer-specific contracts, volume breaks, and manual overrides that could not be reimplemented in the agent.
An ERP with a rate-limited API, no sandbox that mirrored production catalogue data, and a hard requirement that no order be created twice.
A wrong order is materially worse than a slow one, so the system had to be able to say it did not know.
The client’s platform team had to own the result — no arrangement that left us in the critical path after handover.
What had to be true to finish
5 goalsAgreed up front, so the engagement could be called finished on evidence instead of opinion.
Move the majority of inbound purchase orders into the ERP without human rekeying.
Reduce order-entry errors rather than trading them for a faster route to the same mistakes.
Route uncertain orders to a human with the evidence attached and a one-click path to correction.
Make quality measurable, so catalogue, prompt, and model changes can be regression-tested rather than argued about.
Hand over a system the client’s own team can run, extend, and debug.
Industry context
Industrial Distribution
AI Implementation
Document Automation
Agent Platforms
ERP Integration
Industrial distribution runs on high order volume, thin margins, and catalogues large enough that no representative knows them completely. Orders arrive in whatever format the customer prefers — email text, PDF, spreadsheet, a scan of a fax — and every one of them is rekeyed by hand, which is where both the cost and the errors live.
Approach
How the work was sequenced.
We built the agent as a durable workflow rather than a single model call: extract, resolve, validate, decide, then write back — with every step traced and every decision reversible. The design principle throughout was that the system must be able to abstain. Confidence thresholds were tuned against a labelled evaluation set, not chosen by feel, and the first production cutover deliberately ran at a conservative threshold so the straight-through rate could be raised on evidence.
Delivery track
Segment width reflects the number of workstreams inside each phase — where the engagement actually spent its effort.
Phase 1 — Evidence and evaluation set (Weeks 1–3)
4 workstreams
Sampled 600 historical purchase orders across the full format distribution, including the awkward tail, and labelled them line by line.
Built the evaluation harness first, so every later change could be scored against a fixed set before anything shipped.
Profiled the product master and quantified how often free-text descriptions resolved cleanly, ambiguously, or not at all.
Agreed the acceptance bar with operations: extraction accuracy, straight-through rate, and a hard ceiling on incorrect orders created.
Phase 2 — Extraction and catalogue resolution (Weeks 3–7)
4 workstreams
Combined layout-aware document parsing with an LLM normalization step to produce a structured order regardless of source format.
Built SKU resolution as a hybrid: lexical search over the product master, embedding search for descriptive matches, and a customer part-number alias table learned from order history.
Emitted a calibrated confidence score per line and per order, tuned against the labelled set rather than assigned by hand.
Made abstention a first-class outcome — the agent returns "unresolved" with candidates attached instead of picking the closest guess.
Phase 3 — Validation, orchestration, and write-back (Weeks 5–9)
4 workstreams
Implemented the order lifecycle as a durable workflow with idempotency keys, so an ERP timeout can never produce a duplicate order.
Called the existing pricing and availability services rather than reimplementing contract pricing inside the agent.
Added deterministic validation rules — minimum order quantities, unit-of-measure mismatches, discontinued SKUs, credit holds — ahead of any write-back.
Cut over to production in week 9 at a conservative confidence threshold, with every automated order reviewable after the fact.
Phase 4 — Review console and threshold tuning (Weeks 8–12)
4 workstreams
Shipped a review console that shows the source document alongside the extraction, highlighting the field that triggered the escalation.
Fed every human correction back into the evaluation set and the alias table, so the same ambiguity does not escalate twice.
Raised confidence thresholds in four measured steps as the corrected-order rate stayed inside the agreed ceiling.
Added per-customer format handling for the six templates that generated most of the remaining escalations.
Phase 5 — Handover and operations (Weeks 12–14)
4 workstreams
Wrote runbooks for the failure modes that actually occurred during the pilot, not the ones imagined at design time.
Ran two failure-injection exercises with the client’s platform team: ERP unavailability and a catalogue import that broke SKU resolution.
Set up alerting on straight-through rate, escalation backlog, and corrected-order rate rather than on infrastructure metrics alone.
Transferred repository ownership, on-call, and the release process; we were out of the critical path from week 14.
What the team kept
Artefacts handed over at the end of the engagement — owned and operable by Undisclosed Industrial Distributor without us.
Production order-entry agent with durable workflow orchestration and idempotent ERP write-back
Human-in-the-loop review console with source-document evidence and one-click correction
Labelled evaluation set of 600 orders plus an automated regression harness wired into CI
Customer part-number alias table built from order history and extended by human corrections
Terraform modules and CI/CD pipelines for all environments
Operational runbooks, alerting, and a tested on-call rotation owned by the client
Threshold-tuning guide so operations can trade straight-through rate against review load themselves
Results
What actually moved.
The agent went live in week 9 at a deliberately conservative threshold and reached 68% straight-through by week 14. Order-entry errors fell by more than three quarters, and the customer service team moved from rekeying orders to handling the exceptions that genuinely needed judgement.
Outcome ledger
Every figure we measured on this engagement, including the ones that are ranges rather than headlines.
Orders entered without human touch
Started at 31% at the week 9 cutover; raised in four measured threshold steps.
0%
68%
Median order-entry handling time
Blended across automated and escalated orders.
9m 40s
2m 10s
Order-entry error rate
Wrong SKU or wrong quantity reaching fulfilment.
2.7%
0.6%
Line-item extraction accuracy
Measured on 600 labelled historical orders spanning all formats.
99.2%
held-out evaluation setCost per processed order
Fully loaded labour plus inference and infrastructure.
$4.10
$1.35
Time to first production cutover
Full engagement ran 14 weeks including handover.
9 weeks
from kickoffDuplicate orders created
Idempotency keys on every ERP write-back.
0
since cutoverWhat changed day to day
The part of the result that never shows up in a dashboard, but is the reason the numbers held.
Customer service representatives moved from rekeying to exception handling, absorbing a seasonal volume peak without temporary staff.
Order acknowledgements went out the same day rather than the next, which removed a recurring source of customer escalations.
Fewer wrong-SKU shipments meant fewer returns, credits, and expedited replacements downstream.
The platform team can raise or lower the straight-through rate themselves as confidence in the system changes.
The labelled evaluation set became a durable asset — the next model upgrade was a scored decision rather than a leap.
Your turn
Run this play with your team.
A 14-week implementation put an order-entry agent into production that reads inbound purchase orders, validates them against the ERP, and writes 68% of them straight through. If that shape looks familiar, the first conversation is a working session, not a pitch.
AI Implementation & Production Delivery
Document Extraction & Normalization
Agent Orchestration (LangGraph)
ERP Integration & Write-Back
Human-in-the-Loop Review Tooling
Evaluation Harness & Regression Testing
What Undisclosed Industrial Distributor got
Orders entered without human touch
0% → 68%
week 14
Median order-entry handling time
9m 40s → 2m 10s
measured at week 14
Order-entry error rate
2.7% → 0.6%
rolling 30 days