Section 18 · Testing, Reporting & Learning
18.4Drop-off analysis
Designed for Phase 2- No.
- 18.4
- DMA's reference
- Decision Book p.33
- Phase
- Phase 2
- Autonomy
- L1 · Recommend
- DMA's decision
- Yes
What DMA asked for · Decision Book p.33
Find the exact point where good prospects abandon the conversation.
Why it matters to DMA: It tells DMA exactly what to fix in the conversation.
In our words, from DMA's Product Decision Book v3, page 33.
How we do it
- Scheduled for Phase 2, once the chat is live on DMA's site.
- Each chat already records its stage as it goes: opened, engaged, reviewed (site review shown), booked, or handed to a person.
- The design turns those stages into a funnel, and for chats that stopped, shows the last AI message before the visitor left, so the weak step is visible.
Already in place
- Every website chat records its stage as it moves: opened, engaged, reviewed, booked.
- Every message is stored, so the last line before a drop is known.
What the later phase adds
- The stage funnel with drop-off rates.
- The last AI line before each drop, grouped.
The flow
It plays on its own while it's on screen; hover or use the controls to pause or step through.
The stages
Each chat moves through the stages.
- Stored data
- Screen
Where the data goes
The same six stages on every page. Nothing reaches Nutshell except through the write gate.
- 1Website chat
Source
Stages and messages.
- 2Sync
Copies Nutshell changes into the bridge database
Not involved.
- 3Bridge database
A copy of the CRM data, plus what the AI works out
The stage of every chat and its messages.
- 4AI
Claude models, only through DMA's own gateway
Could group the last lines by theme.
- 5Write gate
The only way back into Nutshell: checked, approved when needed, sent once
Not involved.
- 6Nutshell
The system of record
Unchanged.
The tables behind it
Drawn from the POC's database catalogue: structure only, no data.
PK primary keyFK reference the database enforces (solid line)ID reference kept by id (dashed line)
| Table | What it holds | Columns |
|---|---|---|
| conv.conversations | Every chat with its current stage. | 22 |
How the tables connect (1)
| Column | Points to | Kept by |
|---|---|---|
| conv.conversations.agent_version_id | agent.agent_versions | The application (by id) |
Worked example
Synthetic demo data: every name, business and number is made up.
The chat stages recorded in the POC today (test chats, live counts), as the funnel would read them.
Chats by furthest stage
| Stage | Chats |
|---|---|
| Opened only | 12 |
| Engaged | 8 |
| Reviewed (site review shown) | 3 |
| Booked a call | 9 |
In the running POC
There is no screen for this one yet: it is designed for Phase 2, as DMA's book decides. The parts listed under “Already in place” run in the POC today.
See it live
Opens the exact screen in the running POC, in a new tab.
| Open | Where | What to try |
|---|---|---|
| Live chats | DMA AI dashboard | Chats happening now, with their stage. |
Status
Designed for Phase 2. DMA's book schedules this for Phase 2. The design is ready and builds on parts that already run in the POC.