Skip to content
View as .md

LLM models and keys

Users pick models, never providers. A model ref is provider/model-id (pi format, e.g. openrouter/anthropic/claude-haiku-4.5); the provider prefix only decides which key to look up. OpenRouter is the default: one key covers every model namespace. Direct-provider refs (anthropic/<id>, google/<id>) work only if a key for that provider resolves.

Model resolution

For bot replies and coding runs, the first model configured wins, and it always runs as configured:

(coding runs: environment coding model) → agent `model` → thread config `model` → house `llm.model`
nothing configured → `openai/gpt-6-luna` if the asker connected ChatGPT, else `DEFAULT_BOT_MODEL` (openrouter/z-ai/glm-5.3-flash)

The prefix is the route, as in pi: openai/<slug> runs on the asker’s ChatGPT plan when the plan serves that slug, else on OpenRouter’s openrouter/openai/<slug> with the house key; any openrouter/… ref always runs on OpenRouter, so openrouter/openai/<slug> pins OpenRouter on purpose. A bare vendor ref like z-ai/glm-5.3-flash means OpenRouter. chatgpt/<slug> is an old alias of openai/<slug>, and a bare OpenAI name like gpt-6.1-sol means openai/gpt-6.1-sol. arbe thread trace shows the model on each reply and the route on each completed turn (on Oskar's ChatGPT plan or on the house key); see pi auth.

Agent = durable preference. Thread = “try this model for this conversation”, set rarely. OpenRouter ids resolve from pi-ai’s bundled catalog first, then OpenRouter’s live catalog, so a new model needs no arbe release; an unknown id still fails loudly. The director’s decision pins its own model id (DIRECTOR_DECISION_MODEL_ID in packages/core/dispatch/reply.ts) and takes only the provider from the thread.

Code: resolveReply in packages/core/dispatch/reply.ts; catalog in packages/core/pi/openrouter-models.ts.

Key resolution

One precedence for replies, director decisions, titles, cron parsing and delegated coding agents:

the asker's ChatGPT plan → house BYOK secret → funded house key → clear refusal

The plan tier applies to replies and delegated coding runs only, and only when the human the turn bills has connected one (pi auth).

Both live in the same house secret, OPENROUTER_API_KEY: funded_by = 'arbe' means arbe minted it, null means the house brought its own. Nothing on this path ever reads an operator model key. Without either, the refusal is visible in the thread: No model key for this house. Add OPENROUTER_API_KEY under house secrets. (HTTP: agent.not_configured, 400). Turn overlays are cached ~60s, so a rotated key lands within a minute; a failed secret read fails closed and is never cached.

Code: resolveTurnSecrets in packages/core/dispatch/turn-secrets.ts, the one resolver behind every seam.

Funded keys

Creating a house mints it an OpenRouter key under arbe’s account with a $5 lifetime ceiling and stores it as the house’s OPENROUTER_API_KEY. Minting is best effort and never blocks creation; the create response says funded_key: minted | failed | skipped. Setting your own key replaces funding and disables the minted key. Deleting the secret or the house deletes the key.

A few things here look wrong at first glance but are on purpose:

  • The funded key goes into the sandbox. A bot with run_command can read it with env. That is accepted: the key is capped at $5, scoped to one house, and revocable by rotation. The alternative (proxying every sandbox model call through arbe) was rejected as not worth the moving parts for an alpha.
  • The ceiling is not reserved credit. All houses share arbe’s balance, which can run out first. A house at its own ceiling sees budget_exceeded; arbe’s balance running out sees provider_balance_exhausted. Both name BYOK as the fix; BYOK 402s stay plain provider errors. OpenRouter reports both as 402 with no distinguishing body, so arbe asks /api/v1/auth/key for the calling key’s remaining allowance to tell them apart.
  • Rotation is not a refill. arbe house fund rotate --house <id> disables the old key, reads its usage, and mints a replacement with only the remaining allowance.
  • OpenRouter is the real limit; our ledger is just the estimate. The ledger’s cap (houses.spend_cap_usd, assertWorkerSpendAllowed) is a fast pre-check and an optimistic estimate: it undercounts when a worker dies before recording, and OpenRouter lets one request overshoot the ceiling. arbe house fund status --house <id> shows the key’s real usage next to the ledger, with the gap labelled “unexplained”; that gap is a reconciliation discrepancy, not proof of outside use.
  • Owners cannot raise their own cap. Operators run bun run --filter '@arbe/www' fund-limit -- <houseId> <usd>, which updates both the key ceiling and the ledger cap. fund-backfill [--dry-run] [houseId...] mints keys for houses that predate funding.

OPENROUTER_MANAGEMENT_KEY lives only on the www worker, with an optional OPENROUTER_WORKSPACE_ID naming the OpenRouter workspace keys are minted in (default workspace otherwise). It is never a house secret, sandbox variable or backstage secret; self-hosters who leave it unset get no funded keys and must BYOK. Code: packages/core/openrouter-keys.ts, apps/www/src/routes/api/houses/.

Spend attribution

Every paid seam calls recordUsage() (packages/core/usage.ts) with a key_source:

key_sourceWho paysCounts toward spend_cap_usd?
workerarbe (funded house keys or infrastructure keys)yes
househouse (BYOK secret)no
envhouse (env-bound secret)no
planthe person who asked, on their ChatGPT planno

The resolver calls a minted key funded; recordUsage folds that into worker at that one boundary. Sandbox usage looks up the house key’s provenance (cached 60s, failures not cached). Cap checks run only before an arbe-funded spend: replies, director decisions, sandbox, cron parsing, GIF search, paid vision indexing. A BYOK house over cap keeps getting LLM turns; only worker-only seams refuse.

One gotcha worth knowing: budget_check is SECURITY DEFINER on purpose, because usage_events has no select policy and an invoker-rights gate would sum zero rows (migration 20260611150000).

Exceptions

  • Developer model proofs use ARBE_PROOF_OPENROUTER_API_KEY, passed explicitly.
  • File readings of images and scanned PDFs use the backstage’s GEMINI_API_KEY; they count against the house cap and are refused per file when over it. Text indexing makes no model calls (volumes).