Daytona runtime

delegate_task is arbe’s coding tool. It runs a coding agent in the Daytona box bound to an environment.

This doc is the daytona implementation. It runs pi in a Daytona sandbox, writes pi’s session into one durable arbe thread, and resumes a run from that thread. The code is in packages/sandbox/src/daytona/. Each layer has a runnable proof at proofs/prove-*.ts (bunx varlock run -- bun proofs/<file> from that dir; see proofs/.env.schema).

The shape

The thread is the only durable handle, and the source of truth. A thread is a durable stream of entries. Everything else is a cache you can lose — pi’s session files on disk, the in-process sandbox handle, any real-time watcher — and every cache writes to the thread or reads from it.

The parts:

thread — the durable stream and source of truth. Entry types: chat, pi.*, signal.*.
the run’s task — one Absurd task in backstage (coding-run, task id = run id). It is the only thing that claims the thread, launches or ends the run, and reports it to the parent. packages/core/dispatch/coding-run-task.ts.
pi — the coding-agent process.
arbe-pi-runner — drives pi --mode rpc inside the sandbox over stdin/stdout. It decodes pi’s events into pi.* entries (plus a heartbeat every ~5s), steers chats into a busy pi, aborts on a stop, and reports signal.thread.pi_settled each time pi goes idle or exits. It never ends the run. Lives in src/daytona/runner.ts.
arbe-proxy — a Supabase Edge Function the sandbox calls arbe through, because it can’t reach the CF worker. Carries all of /api/*. Covered under egress below.

A run, end to end:

delegate_task (key: its chat) / a human chat after a run ended (key: that run) ──▶ coding-run task
task: claim (row → running, run_id, epoch+1, marker) → box → launch (runner)
      → park ◀── wakes: runner report (proxy) · stop · chat in the child (director)
      → read thread + row → wait | continue (run_continue) | end
      → fence (epoch+1, terminal entries) → tell the parent → close the row → next run?

Events are only wake-ups; the task re-reads the thread on every wake and at least once a minute, so a lost wake only delays. Each park waits on a fresh event name (coding-run:<run>:<generation>, the generation kept on the row as run_wake), because Absurd events are first-write-wins per name and its awaitEvent takes one name with a timeout. Every step is checkpointed: a backstage restart resumes the task where it was. The box is its own step before the runner fires, and a per-run box carries an arbe.run label, so a launch redelivered mid-way finds the same box instead of cutting a second.

The task decides from the thread alone (decideCodingRun in @arbe/core/pi/coding-run):

runner busy, beating                         → wait
runner silent 90s since launch / last report → end failed (pi_session_orphaned)
stop, runner busy                            → wait for its report, end cancelled after 60s
runner idle, its own unstopped chat untaken  → continue with those chats
runner idle otherwise                        → end: cancelled after a stop, else pi's outcome

Liveness is measured only on the current run’s entries, from its own launch, and only the runner’s agent speaks for it.

Working with a run

The child thread is a conversation with pi, not a one-shot.

Delivery is recorded. When pi takes a message in (its user message_end), the runner appends signal.thread.pi_input naming the chats and holding the message pi actually received (or handled when an extension consumed it), in order with pi’s own entries; a fresh-box rebuild replays that message. What pi still owes is every chat with no such record, said after the last stop (pendingChats in @arbe/core/pi/coding-run). The delegated task is posted as a chat before the run starts; the runner reads it back as pi’s prompt.
One payer. A run takes work only from the bot that delegated it and the human it bills; both are on its running marker (delegatorId, payerId), and joinsRun in @arbe/core/pi/coding-run is the one rule the first prompt, steering and continuation all use. Anyone else’s chat waits: the task names it in a signal.thread.chat_waiting, a stop leaves it alone, and once the run ends it starts a run billed to its author (their plan if connected, else the house key). The same holds for a house-key run, so a human’s work always runs on their own account.
```
chat in a run's thread ──▶ author is the delegator or the run's payer?
   yes ──▶ joins the live run (first prompt · steer · run_continue)
   no  ──▶ chat_waiting ──▶ run ends ──▶ nextCodingRun ──▶ new run, payer = author
```
Steer. A chat that joins the run, posted while pi works, reaches it as an RPC steer, taken in before pi’s next model call.
Idle. pi’s agent_settled (or, after a command pi handled itself, get_state saying it is not streaming) makes the runner report pi_settled with pi’s outcome as data. The task then either posts signal.thread.run_continue naming chats that came in meanwhile, which the runner prompts, or ends the run.
Stop. POST /api/threads/:id/stop (arbe thread stop, the readout’s stop button) appends signal.thread.stop_requested and wakes the run’s task. The runner counts the stop the moment it arrives, so a continuation it was already reading for is dropped rather than prompted. On a busy pi it clears pi’s queue, sends abort straight to pi (control never waits behind transcript writes), aborts again if a prompt still in pi’s preflight starts anyway, and kills pi’s process tree after 30s. On a parent thread it stops every coding run that thread delegated. A stop drops what was waiting; a chat after it is new work.
Ending. The task fences the run (fence_coding_run: epoch+1 once, run_ending), writes the terminal at the new epoch, posts the parent’s child_finished and handover chat (producer handover:<run>, so a retry lands once), and only then closes the row. No later run can claim the thread before that report lands. The runner sees its terminal in the tail and shuts pi down.
Follow-up. A chat in the child wakes the thread director, which hands a coding-run thread to nudgeCodingRun: wake the running run’s task, or spawn the next run. One rule (nextCodingRun) decides that for the director and for the ending task alike: a human chat not taken in that no run begun after it takes work from, once the newest run has ended. The next run bills that chat’s author. Both spawn with the run it follows as the key (coding-run:<thread>:after:<run>), so they can only start the same next run, and a chat a failed run never took does not block later ones.
The run record. The thread row’s run_id, run_epoch, run_ending and run_wake belong to the task. Every runner write goes through producer coding-run at the run’s epoch, so the stream refuses (403) an older runner and deduplicates a retried batch. A relaunch of the same run after a crash finds its runner alive (runner-<run>.pid) and does nothing, and never re-hydrates a session already on the box. A new run waits up to 3 minutes, heartbeating, for an earlier run’s runner or pi to leave the box, and never kills it; if it is still there, the run reports failed without starting pi. A runner none of whose writes landed for 60s kills its own pi, which usually clears the way, but the 90s silence its task measures runs on another clock.
One attempt acts on the box. Absurd gives a task whose claim lapsed to a new attempt while the old one keeps running. So the task renews its claim right before every change to a box (create, install, upload, fire); an attempt that lost it stops there, and a box it already cut is left for the new attempt to find by its run label.
Idle reports land. A report that cannot be written is retried whole until it lands; meanwhile the runner stops heartbeating, so its task sees silence rather than a healthy idle runner.
No run without an owner. Backstage’s minute sweep (adoptOrphanedCodingRuns) finds rows still running whose task is gone or finished (coding_run_orphans) and spawns an adopting task keyed coding-run:<thread>:adopt:<run>. It takes the run over without relaunching, keeps a terminal already written, and otherwise ends the run failed if the old task failed, then reports it.
Narrow tokens. A run gets two: run appends to its own thread as its producer and reads it; tools calls arbe’s tools as the delegating bot and touches no stream. No sandbox token can PUT, DELETE, or write run control (run_state_changed, pi_failed, run_continue). Tokens last two hours and renew at POST /api/stream/:name/renew while the row still names their run as running. pi’s arbe-tools extension reads the tools token from $ARBE_TOOLS_TOKEN_FILE, a per-run file the runner replaces atomically on renewal. On the payer’s ChatGPT plan the tools token also names the payer, and pi’s model calls go through arbe with it; the box holds no OpenAI credential (pi auth).
Seeing it. The task is a row in absurd.t_default (task_name = 'coding-run', task_id = the run id on the thread row, or params.adopt.runId for an adopter); its checkpoints (claim, box, launch, end, fence, handover, close, next) are in absurd.c_default. See debugging.

Resume continues the same pi session:

  same sandbox   { resume }           pi --continue (the box's own session)
  fresh sandbox  { resume, hydrate }  rebuild session from thread,
                                      upload, pi --session <file>

A fresh box rebuilds pi’s session from the thread with rebuildPiSession (core): assistant messages (text, thinking, tool calls) and tool results as written, and user messages where pi took them in, so a steer sits after the tool results it waited for. Compaction summaries are not replayed; pi gets the full history and compacts again if it needs to. pi adds its own system message on the first request. Loaded with --session, not --continue, which scans a directory filtered by cwd and raced by mtime.

Library (`src/`)

thread.ts — the write side. openThread() mints a scoped grant and returns the stream endpoint; create, post, and read delegate to @arbe/core/entries.
sandbox.ts — the compute side. createSandbox() returns a fresh Daytona sandbox with .exec and .provision().
pi.ts — pi’s argv (--mode rpc, model, session flags).
launch-coding-agent.ts — launchCodingAgent(), the daytona body of delegate_task: post the task (or not, for a follow-up), fire arbe-pi-runner detached, return the thread id at once. Its default thread is an openThread() grant and stream with no DB row; the parented child row with environmentId is core createThread’s job.
runner.ts — arbe-pi-runner, running in the sandbox, with pi-rpc.ts (the JSONL client), thread-writer.ts (batched entry writes), thread-tail.ts (long-poll follow), credential.ts (token renewal) and proc.ts (process-tree kills). How a pi turn ended is decided by core’s decideAgentOutcome (packages/core/pi/outcome.ts), the same function in-thread replies use; a run refused at startup and the silence watchdog decide on their own.

Egress and env

Daytona egress is whitelist-only and matched by domain. The sandbox can reach npm, github*, cloudflare.com, and our Supabase host *.supabase.co (plus the other default-allowlisted services: package managers, git hosts, container registries, LLM APIs). It cannot reach the CF worker arbe.0sk.ar, workers.dev, or any other host, and a tunnel doesn’t help because the block is by domain. At our org tier the restriction cannot be overridden per sandbox; lifting it means Daytona tier 3 (~400 EUR/mo), which we’ve decided against. So: the open web is reachable only via a proxy on an allowlisted host, if a workload ever truly needs it — none does today, since the harness itself only needs the allowlist (arbe-5783). The signature of a blocked host: HTTPS gets a TLS reset (curl exit 35, 000 status), plain HTTP gets a proxy 403.

Don’t take any of that on faith — measure it:

bunx varlock run -- bun run scripts/sandbox-egress-probe.ts <daytona-sandbox-id> [--url <extra>]

It runs curl inside a real box and prints REACHED/BLOCKED per host. Run it before building anything that calls out from a sandbox, and to re-confirm the claims here rather than trusting a doc that has had time to go stale.

That is why sandbox→arbe traffic runs through a shim. *.supabase.co is reachable, so the proxy is deployed as a Supabase Edge Function at supabase/functions/arbe-proxy. It forwards /api/* verbatim — method, path, query, body (binary-safe), and the caller’s own Authorization/Content-Type — to arbe.0sk.ar, and refuses anything outside that prefix. One secret: ARBE_APP_URL=https://arbe.0sk.ar. Deploy with bunx supabase functions deploy arbe-proxy (config pins verify_jwt=false).

The shim is not a trust boundary and must never become one. Every route behind it is already on the public internet under the same route-level auth; the proxy restores reachability from one room and nothing else. It verifies nothing, mints nothing, rewrites nothing — so the route stays the single place trust is decided. A proxy that starts making decisions is a second, weaker auth surface.

Everything a sandbox calls hangs off one base, API_URL in packages/sandbox/src/daytona/config.ts (override: ARBE_API_URL), and STREAM_URL is derived from it rather than hard-coding a second host.

Three more sandbox facts: coding runs start on our own snapshot (CODING_SNAPSHOT, 4 vCPU, 8 GiB, 10 GiB disk) with Bun, pi (PI_PACKAGE) and gh baked in, while house sandboxes follow Daytona’s managed daytona-small; both request language: 'typescript'. Provisioning still checks pi and gh and installs them if missing. Bumping pi or Bun means a new snapshot name and a rebuild with scripts/build-coding-snapshot.ts; the runtime Daytona key can’t create snapshots, so build with your own login (--print gives the Dockerfile for daytona snapshot create). The sandbox runs as a non-root user, so upload artifacts to a HOME-relative path, not /root.

The runner’s write path:

mint:   mintStreamWriteJwt(houseId, threadId, DURABLE_STREAMS_SECRET) -> { jwt, expiresAt }  (2h TTL)
write:  POST <url>/api/stream/arbe-thread-<threadId>   Authorization: Bearer <jwt>   Producer-Seq: <n>
        body = [{ id, ts, payload }, ...]              (the proxy sets Producer-Id and Producer-Epoch from the token)
read:   GET  <url>/api/stream/arbe-thread-<threadId>?offset=<o>&live=long-poll   Authorization: Bearer <jwt>

A scoped token can only touch its own thread; a cross-thread write returns 403.

Alongside the plumbing below, a box also gets the house secrets its environment binds — run_command exports them onto the command it execs, delegate_task into the environment pi runs under. Bound names only, and never arbe’s own operator keys; the boundary and its two rules are in secrets. Binding is the opt-in, so a GITHUB_TOKEN the house holds but no environment binds reaches nothing.

The runner reads its config from these environment variables, set by launchCodingAgent:

Env var	Required	Meaning
`ARBE_THREAD_ID`	yes	target thread
`ARBE_STREAM_URL`	yes	stream-write base URL — `arbe-proxy` from the sandbox; the CF worker only off-sandbox
`ARBE_STREAM_TOKEN`	yes	scoped `stream:write` JWT (the `jwt` from `mintStreamWriteJwt`)
`ARBE_RUN_ID`	yes	the run’s id, on its `running` marker
`ARBE_PI_ARGV`	yes	pi’s argv as a JSON array; never a shell string
`ARBE_AUTHOR_ID`	no	informational author hint; the stream proxy stamps from the token’s `agent` claim

Arbe’s tools in a run

pi loads arbe-tools.mjs, an extension bundled beside the runner (packages/sandbox/src/daytona/arbe-tools-extension.ts). At startup it lists GET /api/threads/:child/tools and registers each tool with pi; each call is a POST /api/threads/:child/tools/:name that runs the same AgentTool an in-thread reply uses (packages/core/dispatch/coding-run-tools.ts), as the delegating bot. A tool is offered when it carries codingRuns: true and the bot’s allow and deny lists, in both the run’s thread and the one that delegated it, still permit it. pi records the calls as its own, so they show in the child thread and in arbe thread now with no new entry types.

The route accepts only the run’s tools token, and only while the thread row names that run, at that epoch, as running. A side-effect tool re-checks that right before it acts. A call whose response is lost is re-sent only when the tool is replay-safe; otherwise pi gets “may or may not have run”. post_to_thread refuses the delegating thread (the handover goes there). The token is readable from pi’s bash, so the run’s three-post allowance is counted on the server in coding_run_posts, keyed by run id, never from what the box reports. A paid nested call (ask_thread) bills the run’s own thread; the thread it read is kept in the event’s meta.readThreadId. arbe_docs is not offered to runs.

Env var	Required	Meaning
`ARBE_API_URL`	no	arbe API base through the egress shim (`API_URL`)
`ARBE_TOOLS_TOKEN_FILE`	no	per-run file holding the tools token; renewed by the runner

Sandbox files stay in the sandbox

Finishing a run publishes nothing. arbe-pi-runner never reads the working tree after pi settles and never sends a volume request; files the coding agent wrote stay on the box, reachable through the normal sandbox workflow (arbe sandbox, run_command, a resumed session). Putting a file in the house is the agent’s own explicit act during the run, through write_file. delegate_task accepts no file list: sandbox work and house-file publication are separate actions.

Box lifecycle

A sandbox is a machine, with a lifecycle independent of any run. pi is one process inside it; many threads, commands, and runs can share a box, and the box outlives any single run. A pi terminal ends a process, never the machine — box teardown is never keyed on a run’s terminal, and never on a wall-clock timeout against the run.

There is no run length. A coding agent runs until it finishes; ARBE_PI_TIMEOUT (DEFAULT_RUN_TIMEOUT_SECONDS, 2 hours) is a runaway guard for a hung process.

Reaping is arbe-owned and idle-based. A box stays up while any thread on it (threads.sandbox_id) is running; once none is, the run’s task deletes it as its last step, and the reconcile/prune sweep (reconcileStuckThread / pruneStuckThreads) catches boxes of other stale threads. reapSandboxIfIdle first retires the row (retire_idle_sandbox: under the row lock a run’s claim also takes, it checks no thread runs there and marks it dead), then deletes the box via an injected reapBox, so @arbe/core stays provider-free. A thread already on the box cannot claim it on its way out. Resume re-resolves to the environment’s live box, or makes a fresh one, so idle means delete; a stopped box buys nothing. The discriminator is sandboxes.ephemeral: a replacement box created only to resume an old child is reapable; a box attached to the parent for later delegated work and an environment’s shared run_command box are retained.

Death is pull-confirmed, never webhook-driven — one org-wide webhook is a forged-event risk against other houses’ boxes. confirmSandboxLiveness(sandboxId, probeBox) asks Daytona about a house-scoped provider_ref through that house’s own runtime: a missing box (404) or terminal-fault state (error/build_failed/removing) tombstones the sandboxes row to dead and orphans the non-coding-run threads on it (a coding run’s task notices its silent runner itself); a live/unknown verdict is a no-op, so a network blip never tombstones a real box. It rides the reaper’s two seams: the read-path/prune reconcile for the current thread and GET /api/sandboxes?house_id=<id>&reconcile=1 for a cold-row sweep; there is no cron. probeBox is injected by the worker, mirroring reapBox.

Daytona’s own auto-stop/delete is the dumb backstop. createSandbox sets autoStopInterval from the runaway guard plus a buffer (Daytona counts only API calls as activity, and a detached run makes none, so the clock runs from launch — the value must clear the guard or it would stop a live run). Every box is created with auto-delete set to 3 days after it stops (STOPPED_AUTO_DELETE_MINUTES); Daytona’s own default keeps a stopped box, and bills its disk, forever, and 0 would mean delete on stop. Before a detached run reuses an existing box, connectSandbox raises that box’s interval to the same guard-plus-buffer value; otherwise a shared box’s default 15-minute stop could kill a healthy agent. When the run’s task ends and the box is retained rather than reaped, reapCodingRunBox puts it back on the 15-minute idle stop once no thread runs there; leaving it at the guard value kept every used box billing for days. The raised value catches an abandoned box (worker dead, runner crashed) after ~3 hours; it is not the primary reaper.

Every spawned box is labelled arbe.house on create. Boxes dispatch creates also carry arbe.thread and arbe.environment; a box from arbe env create is provisioned with no labels beyond the house one. arbe sandbox list reads arbe’s own records, so it shows each box’s environment; arbe sandbox list --provider shows the daytona view, where the arbe.thread label names the run that made a box.

See dispatch, environments, secrets, runtime.