# One-daemon dispatch

Decision for arbe-9801 (2026-08-22). Status: **fully shipped 2026-08-22** (arbe-228d, arbe-a16f, arbe-e1f9, arbe-448b). Turns and the speaker decision run in the conductor; celld and the inline path are deleted; local dev runs the same dispatcher on a per-dev queue; post_to_thread and wf-step ride the one enqueue. This document is now the historical record of why.

Reached by two routes independently: this session's iteration from the arbe-9801 sketch, and a blind architecture review (fresh model, no access to `.arbe/`, given only the product goal and the observed symptoms) that converged on the same shape. Where they differed, the blind review's version was simpler and won (per-dev queue over origin stamping; keep the existing fence over an advisory-lock rewrite; no interim smarter-fallback).

## Goal

The product goal governs: **group chat that feels fast and human** — you post, the right bot replies quickly without being summoned by ritual, and silence is always explained. The design principle that follows, ranked above per-hop latency and above code reuse:

**Variants of the dispatch pipeline are the primary cost.** Every future special case (a new tool that posts cross-thread, a workflow step, a DM shortcut) must ride the one path, not add another. Debugging "why didn't the bot reply" must never start with "which pipeline was this even on".

## Problem

"Which bot speaks and where does its turn run" has five answers today: the director-cell permit path (prod), the deterministic cell-down fallback (prod), the inline mention-only path (local dev), `post_to_thread` (inline even on prod — ambient bots never wake on cross-thread posts), and `/api/wf/step`'s drive machinery. Decider (celld), driver (conductor), and executor (www on Cloudflare Workers) are three processes on three availability domains, coordinated by two leases, a permit/claim protocol, and a fallback ladder — nearly all of which defends against network failures *between* them.

The observed pains all trace to the split, not to the semantics: ambient untestable locally (dev is a different pipeline), dispatch changes only show real behavior after deploy, long replies die mid-stream at the Cloudflare worker resource limit, and diagnosis starts with pipeline archaeology.

Premise correction to the original arbe-9801 text: a local post does NOT enqueue `director_spawn` for the prod conductor — it runs the inline path in the local worker (proven by `inline-` taskIds / `MAX_INLINE_ATTEMPTS` on its signals). The incident that motivated the task came through a prod-pointed surface. The lockout is the Worker-resident executor plus the shared queue consumer, not the shared DB.

## Decision

**Collapse to one pipeline: www appends and enqueues; the dispatcher decides and speaks; Postgres remembers.**

```
any surface → POST /entries (www, Workers): append entry + director_spawn(queue) → NOTIFY
→ dispatcher daemon (Fly in prod, laptop in dev; bun + @arbe/core):
    claim task → execution_fence_claim (the ONE lease)
    → read stream tail + beat row → decide speaker (core/director pure fns;
      mention/forced path model-free, ambient = one cheap model call)
    → run the turn in-process (createThreadDispatcher, streams via bearer client)
    → commit beat cursor, release fence
Each bot reply enqueues the next trigger — no in-process recursion anywhere.
```

**Survives:** the durable stream as the only log; `packages/core/dispatch/*` (tool loop, turn, signals, reader copy); `packages/core/director/*` pure functions + tests — the locked beat/queue/narration semantics (2026-08-14) port for free; the Absurd dispatch queue, `director_spawn`, the `pg_notify` wake; the Postgres execution fence RPCs; `submitThreadEntries` (always spawn).

**Changes:** the workflow-conductor imports `@arbe/core` and executes decision + turn in its task handlers instead of driving www over HTTP. Verified feasible: everything a turn needs is env-derivable — actor-JWT minting is a secret + `mintAgentJwt`, streams is a bearer HTTP client, `makeDispatchDeps` is a plain function of those. Beat state (decidedThrough cursor, forced queue, sticky decision) becomes one small per-thread row mutated under the fence; thread-director.md itself notes the cell's state is reconstructible from the stream, i.e. celld was never the source of truth.

**Deleted:** apps/thread-director (celld + cell, R2, restart ritual, proofs); the cell-client half of conductor `director.ts` (claim ladder, permits, activation clocks, fallback drive); the `/api/director/*` routes; `/api/dispatch/run` + `/api/dispatch/narrate-failure` and the held-open POST with its timeout pathology; `WorkerResourceLimitError`; `DIRECTOR_SHARED_SECRET`/`DIRECTOR_CELL_URL`/`DIRECTOR_MASTER_KEY`; the inline fan-out, `kickDispatch`, `dispatchMode`, and `deterministicDirectorDecision` as a selection path; the permit schema layer.

Time-to-first-token improves structurally: today's prod path pays queue claim + cell round-trips + a www decision call + fence RPC + a second held-open POST before `signal.dispatch.started`. Target: NOTIFY wake (≤250ms) + fence + one stream read + (mention: zero / ambient: one cheap model call), then streaming. And long replies stop dying at the worker resource limit — a "natural feeling" killer worse than raw latency.

## Local dev topology

`bun run dev` starts Vite through Portless + the **same dispatcher daemon**, pointed at the shared prod DB with `DISPATCH_QUEUE=dispatch-dev-<user>`; the local entries route enqueues to that queue (queue param on `director_spawn`). Deployed www keeps enqueuing to `dispatch`. Prod never depends on a laptop; the prod conductor structurally never sees dev tasks. If one thread gets triggers from both surfaces, the execution fence arbitrates one driver at a time and the beat row is shared truth. Dev daemon down → dev posts visibly queue, and the dev entries route warns when the dev queue has no live worker heartbeat — honest failure instead of a silently different pipeline.

## The arbe-9801 questions, answered

1. **Origin stamping** — superseded. Per-developer *queue name* (queue param on `director_spawn`), not an origin column or claim filter.
2. **Two claimers, one queue** — never happens: distinct queues per surface. No filtered claims, no double-fire window, no adoption. Stale dev tasks sit on the dev queue and are never adopted by prod (local code was the point); the heartbeat warning makes them visible.
3. **Secrets** — the daemon (prod and dev alike) holds DATABASE_URL, LLM keys, service-role Supabase, streams secret, JWT secret. The `BACKSTAGE_SECRET` www seam shrinks toward deletion as the HTTP drives disappear. The "thin conductor" principle is retired *deliberately*: it looked like hygiene but caused the held-open POST, the CF resource deaths, and the dev split. Recorded here as a decision, not drift.
4. **Celld** — deleted, not replicated locally. The deterministic cell-down fallback (a second decision variant) dies with it. Mutual exclusion (fence) + a beat row + Absurd redelivery is the per-thread brain; the missing ingredient was co-locating decision with execution, not the cell.
5. **Blast radius** — a dev daemon claims only its own queue, which only that developer's www enqueues to: local code can only run threads you posted to locally. Data is still prod by construction (there is no local DB); the fence bounds cross-surface races.
6. **DX** — daemon inside `bun run dev` by default; missing secrets fail loudly at daemon boot.

## Rejected

- **Smarter inline path** (real speaker decision in-process in www, dev-only or as the fallback) — a smarter second pipeline is still a second pipeline.
- **Origin-stamped tasks + filtered claims on one queue** — per-dev queues get the same isolation with no filter logic and no orphan-adoption question.
- **Local celld replica** — replicates the topology this decision deletes; cell state authority across laptop/prod is unsolvable by construction.
- **Prod adoption of stale dev tasks** — running dev-posted turns on prod code defeats the purpose; fail visibly instead.
- **Advisory locks replacing the fence** — the fence RPCs already exist, already outlive the cell, and are sufficient alone; no rewrite.
- (Still standing from reply-turn-runtime.md: CF Queues, Durable Objects, hand-rolled claim tables.)

## Risks

- Restart behavior regresses slightly: daemon restart aborts in-flight turns and leans on redelivery + the prior-reply idempotency guard — bounded drain exists; acceptable at alpha.
- The beat state machine ports from celld's actor serialization to lock-under-fence — the one place new correctness bugs can enter; step 2's focused test suite is half that task.
- Daemon trust surface grows (all the secrets) — accepted above.
- Turns occupy daemon slots (IO-bound, concurrency env-tunable) — fine at current scale.

## Migration order

1. **arbe-228d** — turn execution into the daemon (kills mid-stream deaths; cell protocol untouched).
2. **arbe-a16f** — speaker decision in-process; celld deleted.
3. **arbe-e1f9** — dev parity: per-dev queue, daemon in `bun run dev`, inline path deleted. *Ambient works locally here.*
4. **arbe-448b** — `post_to_thread` + `/api/wf/step` ride the same enqueue.