Section 21 · Final Revenue Multipliers & Control Plane

21.10AI cost, traceability & decision observability

Built in the POC

No.: 21.10
DMA's reference: Decision Book p.46
Phase: Phase 1
Autonomy: L4 · Autonomous
DMA's decision: Yes

◆ Amended in v3 of the book.

What DMA asked for · Decision Book p.46

Log the source data, model and prompt version, recommendation, action, cost and outcome for important decisions; use cheaper models for routine work and stronger ones for high-value moments.

Why it matters to DMA: It makes a large AI system debuggable, governable and affordable, instead of an opaque pile of automations.

In our words, from DMA's Product Decision Book v3, page 46.

How we do it

Every model call goes through the AI gateway, which records the worker and its version, the prompt version, the model, tokens, cost, time and a trace id; the prompt and answer are stored encrypted.
Every consequential decision is written to the decision log with the rule result, reasons, tokens and cost, and linked to the model call and the trace, so one chat reply can be opened step by step.
Routine work uses Claude Haiku 4.5 (record reviews, notes, classification, input checks); conversations and writing use Claude Sonnet 5.
A budget, per-caller daily caps and kill switches stop spend; Observe shows it live and the Log page totals it.

The flow

It plays on its own while it's on screen; hover or use the controls to pause or step through.

Step 1 of 3

A call and a decision

Both recorded.

AI
Rule in code
Stored data
Screen

Where the data goes

The same six stages on every page. Nothing reaches Nutshell except through the write gate.

1Every worker
Source
Model calls and decisions.
2Sync
Copies Nutshell changes into the bridge database
Not involved.
3Bridge database
A copy of the CRM data, plus what the AI works out
The call log and the decision log.
4AI
Claude models, only through DMA's own gateway
Haiku 4.5 for routine, Sonnet 5 for conversations.
5Write gate
The only way back into Nutshell: checked, approved when needed, sent once
Not involved.
6Nutshell
The system of record
Not involved.

The tables behind it

Drawn from the POC's database catalogue: structure only, no data.

PK primary keyFK reference the database enforces (solid line)ID reference kept by id (dashed line)

Table	What it holds	Columns
ops.llm_calls	Every model call.	37
ctrl.decision_log	Every decision.	28

How the tables connect (7)

Column	Points to	Kept by
ops.llm_calls.agent_version_id	agent.agent_versions	The application (by id)
ops.llm_calls.run_id	agent.runs	The application (by id)
ops.llm_calls.conversation_id	conv.conversations	The application (by id)
ops.llm_calls.decision_id	ctrl.decision_log	The application (by id)
ctrl.decision_log.agent_version_id	agent.agent_versions	The application (by id)
ctrl.decision_log.run_id	agent.runs	The application (by id)
ctrl.decision_log.conversation_id	conv.conversations	The application (by id)

Worked example

Synthetic demo data: every name, business and number is made up.

One recorded chat, traced

1 / 2

What you see

1.Every call and its cost

The gateway's call list.

localhost:13301/observe

What happens in the system

LLM gatewaythis chat
6 model calls, $0.022 in all.
LLM gatewaythe objection reply
Claude Sonnet 5: 1,673 tokens in, 268 out · $0.0099 · 5.3 s.

Right-hand values come from a real run of the POC on September 25, 2026 (trace tr_muh5tbo52jgpqd0smgrf8, tr_muh5tbqv18g3rgfbfdhda…). Rows marked “sample” are illustrative.

Cheaper models for routine work (averages from the gateway's log).

Model by job

Job	Model	Average cost
Input check on a chat message	Claude Haiku 4.5	$0.0008
Record review	Claude Haiku 4.5	$0.0027
Chat reply	Claude Sonnet 5	$0.0068
Reactivation email	Claude Sonnet 5	$0.0110

In the running POC

localhost:13301/observe

1 / 2Model calls with cost.

1Model and tokens.
2Cost.

Dashboard · Observe → Model calls (LLM gateway). Captured from the running POC on September 25, 2026; synthetic data.

See it live

Opens the exact screen in the running POC, in a new tab.

Open	Where	What to try
Observe	DMA AI dashboard	Live activity and traces.

Status

Built in the POC

Built in the POC. This capability runs in the POC today, on synthetic data, end to end.