# Bot conversation storm proof — S15 full-cast (2026-08-30)

**Scope: this was a focused S15-only run, not a comparable full-suite run.** The suite in
[`tests/director-evaluation-prompt.md`](../../tests/director-evaluation-prompt.md) gained a new
scenario, S15 (full-cast storm: one fresh thread, all seven pinned cast bots, tools disabled in
thread config, a bounded concurrent burst of five unmentioned canned messages, settle to
`skipped flurry_rest` with no manual containment). Only S15 ran today; no other scenario ran, so
nothing here is comparable to a full-suite table. Suite count updated 17 → 18 in the prompt and
the README index. Paid live runs approved by Oskar in advance.

Three runs happened. Run 1 failed its gate (invalid). Run 2 was behaviorally green but is
**superseded**: it ran without preventive tool isolation. Run 3 — fresh thread, tools disabled
before any dispatch, explicit failure containment — is the verdict.

## Run 1 — `klrpowxwmosn`: invalid (gate not met + unexpected file deletion)

Ran against the original S15 wording ("what's one thing this project should stop doing?"). Routing
and termination were correct (deciding beat `-1→4`, nova, `beat_completed`; follow-on beats bix
and ada; `flurry_rest` at 1.4 m; no manual containment), **but the run is not a valid PASS**:

- **Burst gate not met.** The suite requires one captured exit-0 status per background post. The
  poster loop and the `wait` loop ran in different shell invocations, `jobs -p` was empty in the
  second, and all five pid exit statuses were lost. Entry ids (5/5) and trigger lines (5/5) held;
  the exit-status leg did not. Treated as a gate miss, not worked around.
- **Unexpected house-file deletion.** The actionable wording pulled bots into house files: ada ran
  `read_file` / `delete_file` tool rounds and deleted
  `documents/thread-house-signals-mnlltolzxlnm-2026-08-26T15-36-11-929Z.md` in botlab. CLI evidence
  verifies that it and the retained `documents/thread-house-signals-mnlltolzxlnm.md` were identical
  176-byte files, and `signal.files.changed` records the deletion at generation 8. The deleted file
  was redundant, but the write was still an unintended scenario side effect.

Nothing from run 1 counts toward any verdict. Both prompt defects were fixed before rerunning.

## Run 2 — `wtvmnlnnszzt`: behaviorally green, superseded

Fixed single-shell pid capture (5/5 exit 0), safe canned wording (polling-vs-webhooks tradeoff),
zero tool calls, deciding beat `-1→4` ada, follow-on eve and cleo, `flurry_rest` at 47.7 s with no
containment. Behaviorally green — but it lacked **preventive tool isolation**: tools were merely
hoped away by wording, and run 1 had already proven wording alone does not prevent tool use. Not
the valid proof; superseded by run 3.

## Run 3 — thread `ywrymznvrsms` (the verdict)

Fresh thread `ywrymznvrsms` (`arbe-thread-ywrymznvrsms`), name `dir-eval abf6 S15 full-cast-storm
r3`, run tag `abf6`.

**Preflight (third run):** cast check `PASS — cast matches the fixture.` (first line
`endpoint https://arbe.0sk.ar · as oskarrough human a13364ba`); untruncated `arbe wf runs`
production summary `✓ workflow runner healthy — no runs stuck waiting (checked 26s ago)` — the
current healthy form the preflight now accepts; no warning lines, so the four pre-authorized stale
conductor lines (`workflow-conductor-oskar`, `workflow-conductor-781599ebd63278`,
`workflow-conductor-r`, `workflow-conductor-cursor`) never appeared and the one-day override went
unused.

**Tool isolation (before any dispatch).** `arbe config set thread ywrymznvrsms --patch
'{"dispatch":{"tools":[]}}'` — raw proof: `patch: {"dispatch":{"tools":[]}}`; resolved proof
(`config get --json`): `config.dispatch.tools: []` resolved over the house/app chain. No other
config write was made.

### Canned inputs (all five concurrent)

```
storm 1: one line — a tradeoff between polling and webhooks.
storm 2: one line — a tradeoff between polling and webhooks.
storm 3: one line — a tradeoff between polling and webhooks.
storm 4: one line — a tradeoff between polling and webhooks.
storm 5: one line — a tradeoff between polling and webhooks.
```

### Gate — all three counts met before any trace analysis

- **Exit statuses 5/5:** one single-shell invocation spawned all five posts into a pid array and
  waited each captured pid in that same invocation: pids 1406202–1406206, all `exit 0`.
- **Entry ids 5/5**, one complete `{id, threadId}` JSON per post:
  storm 1 `c6c7add6-363f-4c15-95c8-8709d7f97dde` · storm 2 `2460d5a6-3507-41f0-b39a-59ae25b3d835`
  · storm 3 `23aeb41b-1cb5-43c9-9f94-2834ee617c3e` · storm 4 `56707e9e-fb88-44b6-a204-10df8f83ff3d`
  · storm 5 `af054f47-f3d1-4c84-81da-4cf83420db74`
- **Trigger lines 5/5** in `arbe thread trace ywrymznvrsms` (arrival order 2, 3, 4, 5, 1 within
  423 ms).

### Participant snapshot

At creation and after the run — identical, delta empty, no joins, no removals: nova `555539bb` ·
ada `87269073` · eve `ec861612` · bix `ecf8b358` · dot `e483bd47` · cleo `a464541f` ·
probe2-chess `741fbea7` · oskarrough (human, trigger author).

### Trace ranges and closers

4 `beat_started`, each with exactly one closer, offsets contiguous, no gap, covering all five
posted entries (`-1→4` spans all five entry ids):

| beat | range | decision | closer | speaker |
|---|---|---|---|---|
| 1 (deciding) | `-1→4` | `@ada 1.5s llm/1.7s total` (paid) | `beat_completed (-1→4)` | ada — full coalescing, one speaker |
| 2 (follow-on) | `4→11` | `@eve 1.2s llm/1.2s total` | `beat_completed (4→11)` | eve (context) |
| 3 (follow-on) | `11→18` | `@cleo 1.0s llm/1.1s total` | `beat_completed (11→18)` | cleo (context) |
| 4 | `18→25` | `nobody no_candidates 1ms total, no llm time` | `director nobody (18→25)` + `skipped flurry_rest` | — |

0 `coalesced` (the offsets are the coalescing proof: one deciding beat spanning all five entries),
0 `beat_failed`, 0 `fallback_*`, 0 `signal.dispatch.failed`.

### Waits, retries, containment

- One wait cycle for the burst: `timeout 120 arbe thread entries read` exit 0, then five 25 s
  `entries list --follow` settle probes (each exit 124 by design, output to file, never piped
  through `tail`).
- `skipped flurry_rest` confirmed in the trace after probe 5 (at 1.1 m); 30 s later the trace was
  unchanged and `thread diagnose --json` shows `stage: completed`, `verdict: ok`.
- Containment was **not** needed: no sixth probe, no participant removal, no manual intervention.
  The failure-containment rule (remove every cast bot, verify no candidates remain, record FAIL,
  stop) stayed unexercised.
- Retries: none.

### Metrics (thread totals, every reply on the thread)

- Replies: 3 (ada, eve, cleo). Input **2,842**, output **1,859**.
- **Reasoning tokens: 1,725** (45 ada + 832 eve + 848 cleo), from
  `entries list --json` → each `pi.assistant` entry's `payload.message.usage.reasoning` — the
  surface exists and is now the canonical report source (the old `n/a` row was wrong).
- Cost: trace reply lines $0.0001 + $0.0003 + $0.0003 = **$0.0007** (usage payload exact:
  $0.00069). Director-decision spend: `n/a` — not exposed at this layer.
- Context growth visible per reply `in`: ada 1,122 (all five storm entries, tools disabled), eve
  1,157, cleo 563.
- **Paid decision aggregate uses total decision time**, not llm time: totals 1.7 s, 1.2 s, 1.1 s
  (n=3, n<5 → raw sorted: 1.1, 1.2, 1.7). Run 2's report had wrongly used the llm split.
- Post→reply for the deciding beat: **7.2 s**. Whole turns: ada 4.2 s, eve 22.8 s, cleo 29.5 s.
- Trace id: `n/a` — in-process director threads expose no `traceId` via `thread diagnose --json`.

### Tool calls and house-state writes

**Zero from this thread.** `entries list --json` contains no `toolCall` content in
`pi.assistant` entries and no `pi.tool_result` entries; all three turns report `toolRounds: 0`;
`dispatch.tools=[]` was resolved before dispatch.
Scope note: this proves no tool or file-change entry originated from this thread — it is
thread-scoped evidence and says nothing about house-wide changes made by other actors. No
participant removal, no config change beyond the one permitted write.

## Verdict (run 3)

**PASS.** Hard invariants: no `INELIGIBLE_SPEAKER`, no `MISSING_TERMINAL` (4/4 `beat_started`
closed: three `beat_completed`, one `nobody` + `skipped flurry_rest`), no `DUPLICATE_SPEAKER` (one
speaker in the deciding beat), no `MALFORMED_DECISION`, no `LOST_OBLIGATION`, no `FLURRY_BROKEN`
(three consecutive bot turns — ada, eve, cleo — then rest). Judgement: `acceptable`, any of the
seven permitted. Termination: **PASS — stopped on its own**; `flurry_rest` at 1.1 m after three
consecutive bot turns, trace unchanged 30 s later, `diagnose` verdict `ok`, participants unchanged.

## Anomalies and DX findings

1. **Run 1 (`klrpowxwmosn`) invalid:** burst gate not met (exit statuses lost across shells) and
   the original wording drove ada's tool rounds and deletion of a CLI-verified duplicate 176-byte
   botlab house file. Kept as evidence of both failure shapes.
2. **Run 2 (`wtvmnlnnszzt`) behaviorally green, superseded:** no preventive tool isolation —
   wording is not a control. `dispatch.tools=[]` is now the S15 precondition.
3. Preflight now accepts the current healthy production summary
   (`✓ workflow runner healthy — no runs stuck waiting (checked <n>s ago)`); the four
   pre-authorized stale conductor lines never appeared in any run today; the override went unused.
4. The corrected single-shell pid-capture pattern worked first try in both runs 2 and 3.

## Files

- `tests/director-evaluation-prompt.md` — S15 (tools-off config proof before dispatch, safe
  wording, thread-scoped no-tool pass condition, failure containment that bounds spend, suite
  count 18); shared S12 burst snippet (single-shell pid array, per-post exit statuses); Do-not
  exception for the one S15 config write; preflight healthy-summary wording; reasoning-tokens
  report source corrected to `payload.message.usage.reasoning`.
- `tests/README.md` — index line: eighteen scenarios, storm subcase named.
- `docs/thinking/bot-conversation-storm-proof-2026-08-30.md` — this report.

Threads, in run order: `klrpowxwmosn` (invalid: gate miss + file deletion) · `wtvmnlnnszzt`
(behaviorally green, superseded — no tool isolation) · `ywrymznvrsms` (valid verdict).