Section 5 · Reactivate Old Leads Now

5.7Safe staged rollout

Built in the POC

No.: 5.7
DMA's reference: Decision Book p.16
Phase: Phase 1
Autonomy: L1 · Recommend
DMA's decision: Yes

What DMA asked for · Decision Book p.16

Start with a small batch, compare results, protect the email reputation, and scale only when proven.

Why it matters to DMA: It is the risk control that makes everything above safe to turn on.

In our words, from DMA's Product Decision Book v3, page 16.

How we do it

Outreach goes out in waves. Each wave keeps back a holdout (20% by default), matched by tier and segment, that gets no email, so the lift is measured against people who weren't contacted rather than guessed.
Sending is capped: a daily cap (50 in the prototype), per-inbox limits and a warm-up ramp, all DMA settings. Every email waits for approval in the early waves.
Automatic pauses protect the sending reputation: a bounce rate above the threshold pauses outbound email, and the AI budget has its own switch. The owner, the president and operators can pause everything from the header, on every page.
Measurement shows each wave against its holdout: replies, meetings, pipeline and won revenue, with the lift, its range and a plain verdict; simulated history is labelled as such.

The flow

It plays on its own while it's on screen; hover or use the controls to pause or step through.

Step 1 of 4

A list, and a holdout

20% of the wave, matched by tier and segment, gets no email.

Stored data
Rule in code
Person
Routing
Screen

Where the data goes

The same six stages on every page. Nothing reaches Nutshell except through the write gate.

1The ranked list
Source
Accounts chosen for a wave.
2Sync
Copies Nutshell changes into the bridge database
Outcomes (meetings, deals) come back from Nutshell.
3Bridge database
A copy of the CRM data, plus what the AI works out
Waves, holdout assignments, outcomes, send counters.
4AI
Claude models, only through DMA's own gateway
Not involved in the controls.
5Write gate
The only way back into Nutshell: checked, approved when needed, sent once
Holdout accounts are excluded from every sender.
6Nutshell
The system of record
Holdout accounts never hear from the AI during the wave.

The tables behind it

Drawn from the POC's database catalogue: structure only, no data.

PK primary keyFK reference the database enforces (solid line)ID reference kept by id (dashed line)

Table	What it holds	Columns
ctrl.experiments	Each wave's experiment: the holdout share, when it started, its window.	17
ctrl.holdout_assignments	Which accounts were emailed and which held out, by tier and segment.	10
ctrl.kill_switches	The pauses, including the automatic ones.	9

How the tables connect (2)

Column	Points to	Kept by
ctrl.holdout_assignments.experiment_id	ctrl.experiments	The database (foreign key)
ctrl.holdout_assignments.decision_id	ctrl.decision_log	The application (by id)

Worked example

Synthetic demo data: every name, business and number is made up.

Measuring a wave against its holdout

1 / 3

What you see

1.Each wave and its holdout

Measurement lists every wave with the emailed and held-out groups; simulated waves say so.

localhost:13301/measurement

Measurement: each wave against its holdout.

What happens in the system

Bridge databasectrl.holdout_assignments
The holdout is chosen by tier and segment, so both groups look alike.
Rule engineholdout exclusion
Every sender checks the holdout before sending; held-out accounts are never emailed during the window.

Two simulated past waves in the prototype (labelled as simulated everywhere).

Waves

Wave	Emailed	Held out	Status
Win-back wave 1, June 2026 (simulated)	88	22	Window closed
Win-back wave 2, July 2026 (simulated)	88	22	Window closed

The limits

Daily send cap: 50 (prototype)
Undo window: 10 minutes
Automatic pause: Bounce rate above 5% in 24 hours pauses outbound email
AI budget: $50; at the limit, everything pauses

In the running POC

localhost:13301/measurement

1 / 2Measurement: each wave against its holdout.

1Simulated history, labelled.
2A wave: emailed vs held out, lift and verdict.

Dashboard · Measurement. Captured from the running POC on September 25, 2026; synthetic data.

See it live

Opens the exact screen in the running POC, in a new tab.

Open	Where	What to try
Measurement	DMA AI dashboard	Waves against their holdouts.
Controls	DMA AI dashboard	Caps and pauses.

Status

Built in the POC

Built in the POC. This capability runs in the POC today, on synthetic data, end to end.