AURI Finance — an agentic finance department, run live

The premise

Most "AI agent" demos show a model reasoning convincingly about a task. AURI Finance was built to a stricter bar: the agents had to actually do the job, against a real synthetic company's books, with real mistakes seeded in on purpose — and every milestone had to be run live against the Anthropic API and verified against real database state before it counted as done. Bugs found during a live run weren't patched over quietly; they're documented below, because a system that only works in the happy path isn't actually working.

The target company is Meridian Analytics, a fictional B2B SaaS business with six months of transaction history, 120 employees, 87 AP invoices, and 540 AR invoices — deliberately seeded with six realistic anomalies (a duplicate vendor charge, an unexplained margin decline, a payroll-vs-revenue divergence, an AR concentration risk, a budget overrun, and one genuinely ambiguous transaction with no confident answer) that no agent was ever told about in advance.

Meet the department

Each agent has a narrow, real toolset — not a general "do finance" prompt. Some hold write access scoped to their own domain only; several are read-only by design.

Haiku 4.5

Bookkeeping

Categorizes the bank feed, posts only above a 0.75 confidence bar, escalates the rest instead of guessing.

Sonnet · read-only writes

Controller

Validates the trial balance and subledgers, re-runs its own duplicate check, and can only raise a correction request — zero write access to anyone else's data.

Haiku 4.5

Accounts Payable

The only agent with real write access, scoped to its own domain: resolves or dismisses duplicate vendor charges.

Haiku 4.5 · read-only

Accounts Receivable

Prioritizes collections against real aging and concentration risk.

Haiku 4.5 · read-only

Payroll & Workforce

Checks payroll against budget by actual salary account, not a guessed one.

Sonnet

FP&A

Explains margin trends against real GL drivers and sets the next quarter's forecast assumption.

Sonnet

CFO

Delegates to specialists via real nested API calls and synthesizes a schema-validated, fact/hypothesis/recommendation briefing.

Deterministic · no LLM

Orchestrator

Runs the sequence, independently re-checks every gate rather than trusting an agent's self-report.

Deterministic · no LLM

Dashboard Publisher

Assembles the closed period into a real report — and refuses outright unless the period is actually closed.

The control architecture

The books can't close on an agent's word alone. Two independent checks stand between "an agent thinks this is fine" and a period actually flipping to closed:

5 specialist agents run → Gate 1 — independent re-check → FP&A → CFO briefing → Gate 2 — human approval → period closes & dashboard publishes

Gate 1 doesn't trust Controller's own "ready" flag — the orchestrator re-derives it independently. Gate 2 is a hard stop: the only write anywhere in the system capable of flipping period_status.status to closed requires a human to run orchestrator/approve.py by name. Controller itself holds zero write access into any other agent's data — its only lever on a finding is filing a row in correction_requests, confirmed by a dedicated test that asserts the ledger is unchanged after it raises one.

Watch it work: a specialist gets it wrong, and the close still can't complete

The strongest evidence for this architecture isn't a passing test — it's what happened on a fresh database no agent had ever seen, during the M8 end-to-end run.

AP gets a real judgment call wrong

AP's live duplicate-detector run found the seeded $14,200 CloudScale Hosting duplicate pair — and, on its own reasoning, dismissed it as a false positive.

"Two separate legitimate billing items."— AP Agent, live run, dismissing the actual seeded anomaly

Controller catches it anyway

Controller reruns its own independent duplicate scan rather than trusting AP's — precisely so a specialist's dismissal can't quietly become the last word. It flagged the identical pair, filed a correction request, and refused to mark the period ready once the combined unresolved exposure ($28,032.95) crossed the $25,000 materiality threshold.

The close blocks — correctly

Gate 1 failed. Not a bug: the system did exactly what it was designed to do when a specialist's judgment and an independent re-check disagree.

AP corrects itself on the very next pass

Re-run against the same database, AP saw Controller's correction request, correctly reversed the duplicate, and was explicit about what it found and why.

The paid duplicate represents a genuine double-payment needing vendor collections follow-up outside AP's scope.— AP Agent, second live run

Gate 1 passes, a human approves, the period closes

Controller's next pass found trial balance and subledgers tying out cleanly and zero duplicate candidates remaining. A human ran orchestrator/approve.py --approved-by "Jordan Ellis, VP Finance", and period_status flipped to closed for the first time against this database.

The scorecard: 6 of 6 seeded anomalies found

Checked programmatically against real database state and each agent's own submitted report — not asserted from a summary. tools/verify_anomalies.py runs code-level checks where the ledger itself proves it (a row actually marked disputed, a transaction actually escalated) and narrative checks where the evidence is a specialist's own written conclusion.

#	Seeded anomaly	Found	Evidence
1	Duplicate $14,200 CloudScale invoice	✓ found	Marked `disputed` via `resolve_duplicate_invoice`
2	Engineering payroll outpacing revenue	✓ found	Payroll Agent's submitted report
3	Gross margin decline (78%→73.05%)	✓ found	FP&A traced it to hosting COGS growth
4	AR concentration + aging (Vantage, 34%)	✓ found	AR Agent's submitted report
5	Marketing budget overrun (+28%)	✓ found	FP&A's variance scan
6	Genuinely ambiguous $9,800 transaction	✓ found	Escalated for human review, not guessed at

Built by finding real bugs, not by avoiding them

Every milestone in this project ran live against the real API before it counted as done. That surfaced real defects a code-review-only process would have missed. A few of the more interesting ones:

A silently corrupted API call, present since milestone 2 found at M7

All eight agent runners decided whether to send tool results back to the API by checking stop_reason == "tool_use" instead of whether a tool had actually been called. A turn that called a tool but ended for any other reason (usually hitting the token limit) silently dropped that tool's result, corrupting the next API call. It had been there since the second milestone; it only became reproducible once a later agent's larger outputs started brushing the token cap. Root-caused and fixed identically across all eight runners.

The audit trail was missing the "why" for six milestones found at M8

Every agent's terminal report call returned a bare receipt and never wrote its actual conclusion to the audit log — only intermediate tool calls were logged. Building the audit-trail replay tool to prove "who did what and why" is real found that the "why" had never actually been recorded. Fixed with a shared logging call wired into all seven agents.

A control that trusted the exact staleness it existed to catch found at M5

The Controller's "ready" flag was only ever set on success, never cleared on refusal — so after a correct refusal, the orchestrator's independent re-check could read a stale approval from an earlier run and pass anyway. Fixed by clearing the flag on every refusal path, not just success.

A reversal that didn't check whether the money was already spent found at M5

Fixing the bug above surfaced a real one behind it: a live AP run had reversed what it thought was a duplicate invoice that had, in fact, already been paid — creating a permanent break between the ledger and the subledger. Fixed by refusing any reversal against a paid invoice, and by giving AP a way to permanently record a reviewed false positive instead of re-litigating it every run.

An agentic finance department that actually closes the books.