Production Assurance

Go-live is day one.
We keep it right from there.

AI systems drift. Models change underneath them, costs creep and the people who built them move on. We hold every live system to an agreed accuracy floor, catch problems before your team does, and keep the runbooks current, so the system stays trusted long after launch.

Book a health review See how incidents are handled

01 / Premise

Launch is the easy part.
Month three is where systems fail.

A pilot goes live with a demo that works. Then the model provider ships an update, document formats change, and the one engineer who understood the system leaves. Nobody notices the accuracy slipping until a client or an auditor does.

Launch day

The pilot went live with a launch email and a demo that worked.

Go-live checklist — Credit memo assistant

CheckOwnerStatus

Demo to the credit committee

Signed off on 2 June

Sponsor

Done

Pilot accuracy

91% on 200 sample memos

Vendor

Done

Launch email to the desk

Sent to 64 analysts

Sponsor

Done

Monitoring

To be agreed after launch

Not set

Evaluation suite

The pilot sample only

Vendor

Partial

Runbook and on-call

Held by one engineer

Vendor

Not set

6 checks · 0 owners after launch · live

Month three

Three months later, the model changed, costs crept up and the engineer who built it left.

Accountability

The business still relies on it, and the auditor still asks who is accountable.

02 / Health review

In two weeks, we review every AI system already in production, ours or anyone’s, and rank what to fix first by risk to the business.

Fix first

Low

Medium

High

Risk to the business ↑

2 wks46810 wks

Effort to fix →

Accuracy drift

Latency

Model version

Access

Cost per case

Ownership

Credit memo assistant

Credit

Client report drafter

Client service

Research notes

Research

Complaint drafter

Customer ops

Sanctions screener

Compliance

KYC file builder

Onboarding

Risk score, 0.1 to 1.2

0.11.2

Shortlisted

1.2

0.5

0.2

0.3

0.4

0.1

0.2

0.1

0.5

0.2

0.1

0.4

0.3

0.2

0.1

0.9

0.2

0.3

0.1

0.5

0.7

0.2

0.3

0.8

0.1

0.3

0.6

0.4

0.2

0.1

0.3

0.1

Memo assistant · accuracy drift

Report drafter · access

Complaint drafter · cost per case

KYC builder · accuracy drift

Research notes · cost per case

Fix first · live systemsDay 10

Credit memo assistant

Re-baseline and add a nightly suite

High

6 wks

Client report drafter

Remove shared service account

High

5 wks

Complaint drafter

Route simple cases to a smaller model

Medium

4 wks

KYC file builder

Nightly suite, from fix 01

Medium

3 wks

Research notes

Cache repeat sources

Medium

8 wks

Five fixes, ranked by risk and effort

Fix 04 reuses fix 01

Days 1–6

Logs, access lists and run costs read for every live system

→ system inventory

Days 7–9

Each system scored on accuracy, drift, cost, access and ownership

→ scored matrix

Day 10

A ranked fix list and a run plan for every system

→ fix list

Illustrative estate of six live systems. Real reviews score each one on accuracy, drift, cost, access and ownership, with the evidence attached.

First step

Start with a two-week health review.

Book an audit

The fix list is yours, whether we run it or not

03 / First 100 days in production

The first hundred days,
as a run log.

We take over monitoring, evaluation and on-call for your live systems, fix the highest risks first, and leave a runbook your team has rehearsed. Every change is scored against the suite before it ships.

Run log — live systems · Q3

020406080100

Health review

Days 0–14

Monitoring live

Days 8–30

Memo assistant re-baselined

Days 14–42

Model migration

Days 36–64

Cost tuning

Days 56–86

Evaluation and handover

Days 70–100

Day 0

Log

D03Baseline signed: 11 open incidents, accuracy 79% against a 92% floor

D14Review closed. Fix list agreed, on-call rota live

D30Monitoring and nightly suites live in your own tenant

D42Memo assistant back above the floor · 93%

D64Model migrated · scored on the suite before release

D86Cost per case down from £0.71 to £0.38

D100Runbooks rehearsed with your team · 3 incidents open

Open incidents

Against the baseline signed on day 3

Rework hours saved / qtr

Live

0 of 3

Run checklist · Q3

Memo assistant re-baselinedPlanned

Model migrationPlanned

Cost tuningPlanned

Every figure comes from the platform’s own logs, reported to you monthly.

Traceability

Click any incident.
See how it was handled.

Every event in production is detected, contained, fixed and signed off, and each step is recorded. Your risk team and your auditor read the same log we do.

Run log — Credit memo assistant · September 2026

System CMA-011–30 Sep

Auto-playing · click any incident

Handling

Model provider updateDay 4

DetectedProvider released a new model version overnightVersion watch · 04:10

ContainedNew version held back; system pinned to the approved oneChange gate CG-2 · held

FixedNew version scored on the 1,200-case suite: 93.8%, above the floorSuite run S-0904

Signed offRisk owner approved the migration6 Sep · 09:12

Closed · 4 of 4 stepsSame log in the monthly run report

Illustrative month for one system. Names, references and figures are examples.

04 / Across systems

Set up once. Run across every system.
Each system comes under watch faster than the last.

The monitoring, evaluation and runbook pack set up for the first system goes into a shared library. The next system adopts it, adds its own test cases, and is under watch in days rather than weeks.

Illustrative five-system estate. Timings depend on each system’s logs and test data.

Playbooks

Four systems we run,
written up from delivery.

What we measure first, what we build and how each one is checked. Read them before the audit, or bring them to it.

All nine playbooks →

PlaybookMandate GuardrailsInvestment guidelines turned into approved, versioned rules, tested before and after each trade, with every breach and override on record.Read the playbook →PlaybookClient Reporting FlowClient reports built from reconciled books, each figure tied out and each commentary claim checked against attribution before release.Read the playbook →PlaybookResearch IntakeOvernight house-format notes from filings, transcripts and broker research. Every figure and quotation carries a source.Read the playbook →PlaybookInvestor ReportingLetters, statements and DDQ answers drafted from your books, with every figure traced to source before release.Read the playbook →

05 / Value

Accuracy you can bridge,
line by line.

Every system reports against the floor agreed at handover. The monthly run report shows what moved accuracy and why, and the same evidence answers your risk team and your auditor.

Monthly run report — Credit memo assistant · month 6

Cases right first time

% · axis starts at 92

94.2% →94.2%+0.0 pts

96

95

94

93

92

94.2%

+1.2 pts

+0.5 pts

+0.2 pts

+0.3 pts

96.4%

At handover

month 0

Model migration

scored before release

Drift alerts

caught within a day

New formats

added to the suite

Reviewer feedback

fed back weekly

Accuracy

measured at month 6

+2.2 pts on 41,000 cases a quarter≈0fewer cases reworked a quarter

Illustrative bridge for one system. Each line is measured against the floor agreed at handover.

Related work

Measured against the week
before we arrived.

Client names are withheld under mandate. Each engagement reports the baseline instrumented at the diagnostic and the figure at the third monthly review.

Private Credit · Production AssuranceDaily agent-notice reconciliation for a loan operations teamAgent notices were reconciled to the loan system at month end, so breaks piled up. Notices are now matched every day, and each break carries an explanation before close.Breaks open at month end86 → 4Notices matched each day1,900Breaks without an ownerNoneRead the case study →Asset Management · Production AssuranceClient reporting across an institutional bookQuarterly client reports took six people most of a month. Reports are now built from reconciled books, and each commentary claim is checked against attribution before release.Report production18 days → 5 daysReports per quarter640Claims checked against attribution100%Read the case study →

All work →

Not live yet?

Start with a two-week workflows & processes audit.

We map what your teams do today, measure the baseline and rank what is worth building, before anything goes into production.

Book an audit

The process map is yours, whether we build or not

06 / Engagement

Three ways in.
Each one earns the next.

Engagement 012 weeks · fixed fee

The health review

We review every AI system you already run, ours or anyone’s: accuracy, drift, cost per case, access and ownership. You get a ranked fix list and a run plan for each system.

Covers

Every live system

Access

Read-only

Output

Fix list and run plan

Book an audit

Engagement 0230 days · per system

The handover
to production

We take over monitoring, evaluation and on-call, fix the top risks first and agree the accuracy floor each system is held to.

Runs in

Your cloud

On-call

From day one

Floor

Agreed per system

Book a call

Engagement 03Monthly · per system

Ongoing assurance

We hold the floor and drive cost per case down, with a one-page report every month. Your team can take it in-house at any point; the runbooks assume you will.

Report

Monthly, one page

Notice

One month

Exit

Rehearsed handover

Book a call

3264.ai

Keep it right after go-live.

Book a health review

Monitoring, evaluation and runbooks for AI systems in production, held to an agreed accuracy floor.

32 → 64

Go-live is day one.We keep it right from there.

Launch is the easy part.Month three is where systems fail.

In two weeks, we review every AI system already in production, ours or anyone’s, and rank what to fix first by risk to the business.

Start with a two-week health review.

The first hundred days,as a run log.

Click any incident.See how it was handled.

Set up once. Run across every system.Each system comes under watch faster than the last.

Four systems we run,written up from delivery.

Accuracy you can bridge,line by line.

Measured against the weekbefore we arrived.

Start with a two-week workflows & processes audit.

Three ways in.Each one earns the next.

Keep it right after go-live.

Go-live is day one.
We keep it right from there.

Launch is the easy part.
Month three is where systems fail.

The first hundred days,
as a run log.

Click any incident.
See how it was handled.

Set up once. Run across every system.
Each system comes under watch faster than the last.

Four systems we run,
written up from delivery.

Accuracy you can bridge,
line by line.

Measured against the week
before we arrived.

Three ways in.
Each one earns the next.