Analytics

Three surfaces, picked by what the event is — never send the same fact through two of them by hand:

Signals — durable in-app lifecycle record (signal.<entity>.<verb> on a thread).
track() — PostHog mirror of lifecycle events, normally opt-in per agent; content-free operational counts such as house.created are always-on.
recordUsage() — money. Not opt-in — billing attribution. See usage.

Lifecycle — signals + track()

Signals are the durable record: schema-validated signal.<entity>.<verb> payloads written atomically with the mutation. signal.thread.*, signal.thread_director.*, signal.entry.* and signal.dispatch.* land on the thread itself; signal.house.*, signal.files.* and signal.environment.* on the scope’s primary thread. track() is the PostHog mirror — called at the route boundary, fire-and-forget via waitUntil, only when the agent opted in (or always-on when the event is classified operational).

mutation ──┬─► postThreadEntry  signal.house.created   (durable, queryable, in-app)
           └─► track(...)       house.created          (PostHog, always-on operational count)

Adding a lifecycle event is two independent decisions: post a typed signal.* from the core mutation if it should appear on a thread; call track() at the route if it should appear in PostHog. Both, either, or neither. The one rule: anything worth recording durably belongs in the core schema as a typed signal.* — never through track() alone.

The signal vocabulary lives in packages/core/schemas/stream-entries/thread.ts — families house, thread, thread_director, entry, files, environment, dispatch (28 kinds at the time of writing; the file is the list). To add a kind: extend the schema, post from the mutation. Most signals have no PostHog mirror; only signal.dispatch.* is mirrored wholesale (below), and a handful of track() events exist without any signal (agent.created, member.joined, thread.created, user.signed_in, …). Closing that gap deliberately is task arbe-e9b7.

track() (apps/www/src/lib/server/track.ts) auto-injects route, method, request_id, caller_kind, release_sha, ts; caller properties win on collisions before the PostHog sanitiser runs. Lifecycle events are consent-gated per agent (agents.telemetry_opt_in, surfaced at /account/telemetry) unless explicitly classified as content-free operational telemetry. house.created is always-on so the operator can see the platform’s true creation rate; it carries the house id but not its name. The tri-state is resolved in one place — apps/www/src/lib/telemetry-consent.ts — and during alpha an unanswered null, plus signed-out traffic that has no row at all, counts as opted in; only an explicit false opts out. Flip TELEMETRY_DEFAULT_OPT_IN there to go opt-in-only. Privacy line: no content or identity — track() drops content/name/path fields before capture; app ids (UUIDs) are sent raw; the shared transport disables PostHog GeoIP enrichment.

Errors — trackError(...) from www’s server handleError emits always-on server.error operational events. The backstage emits the same event from worker, Postgres, wake, reconcile, event-loop, and fatal-process boundaries. Both send runtime/source, release, error kind or code, and a hash of the message; raw messages and payload content stay in local runtime logs. The opted-in browser client emits client.error for SvelteKit render/navigation failures, window.error, unhandled promise rejections, and explicitly detected client failures, with the same raw-message exclusion. That is the browser’s only product event: apps/www/src/lib/analytics.ts initialises posthog-js with pageviews, pageleave, autocapture, exception capture and session replay all off, and nothing reads feature flags or surveys — so PostHog’s web analytics, replay, and flag surfaces are empty by design, not by accident. The browser distinct_id is a hashed user:<sha256>, the server’s is the raw agent uuid; the two never join.

Verify the browser path end to end: agent-browser eval "setTimeout(() => { throw new Error('probe') }, 0); Promise.reject(new Error('probe2'))" on any page emits client.error for window.error and window.unhandledrejection, signed in or out. For server.error, throw from any route (a temporary throw in a +server.ts under bun run dev is enough). Ingestion lags ~20-60s; match your own probe by message_hash (djb2 of the raw message), not by timestamp.

Current PostHog lifecycle events: user.signed_in, house.created, house.deleted, agent.created, thread.created, member.joined, environment.created, invite.created, invite.claimed, account.deleted, agent.self_deleted, house.volume.written, house.volume.deleted, house.volume.changeset_written. This list drifts — the authoritative one is a query away (Querying).

Dispatch mirror — publishDispatchSignal (packages/core/dispatch/signals.ts) mirrors the signals it is handed to always-on PostHog events named by stripping the signal. prefix: dispatch.started, dispatch.completed, dispatch.skipped, dispatch.attempt_failed, dispatch.failed, and the progress beat dispatch.phase (phase, round). signal.dispatch.session_started and signal.dispatch.pi_waiting are posted through other paths and are not mirrored. Events carry thread, agent, trace, duration, status, and reason; failures are marked as AI errors.

PostHog groups AI work like this:

thread       → session
selected turn or Pi invocation → trace
model call   → generation
 tool call   → child span

A resumed Pi invocation starts a new trace in the same child-thread session. The director call, selected turn, dispatch signals, tools, and usage share one trace id.

Verify a signal landed: arbe thread entries list <thread-id>. Verify the PostHog side with posthog-cli (Querying).

PUBLIC_POSTHOG_KEY      empty disables PostHog
PUBLIC_POSTHOG_HOST     defaults from PostHog
RELEASE_SHA             stamped on track() events and backstage server.error only; falls back to 'dev'.
                        Usage, dispatch and client.error events carry no release identity (see arbe-b676).

Usage — money

recordUsage(event) (packages/core/usage.ts) is the single call for paid usage. Every spending seam calls it after the spend. Fire-and-forget, never throws, two sinks per event sharing one trace_id. Delivery is best-effort: there is no outbox and no retry, so a sink or process failure drops that event silently — the guarantee is one logical emission per spend, not one durable row. Durable idempotent delivery is deferred to task arbe-e5b3.

usage_events (Postgres, append-only) — the ledger and future enforcement source. Columns: house_id, agent_id, thread_id, capability (llm/sandbox/file_index, open set), seam (open string; live values are whatever ast-grep -p "seam: '$S'" finds — currently director, reply, read_thread, ask_thread, thread_title, cron_parse, sandbox_reply, run_command, delegate_task, sandbox_provision, sandbox_exec, file_search, volume_vision, gif_search; retired values like gate and file_upload stay in PostHog forever), key_source (worker = arbe pays, house/env = the house pays — the resolver merges house secrets and environment bindings into one overlay, so an env-bound key currently reports house and nothing emits env), model_ref, input_tokens, output_tokens, cost_usd (pi-ai’s calculateCost against its own model catalog rates — an estimate, not the provider’s reported charge; llm meta says which via costSource), meta.
PostHog (packages/core/usage-posthog.ts) — $ai_generation for LLM spend (feeds the AI Observability dashboards; no prompt/output content), arbe_usage for the rest. House attached as group, joined to the ledger row by $ai_trace_id. The AI span also carries $ai_latency and a parsed projection of meta: metadata reaches PostHog only when the whole object satisfies DirectorUsageMetaSchema (packages/core/schemas/director-usage.ts — bounded enums, booleans, finite nonnegative integers, no unknown keys). A poisoned or unknown bag projects nothing at all rather than leaking by default, and meta.failed becomes $ai_is_error.

Each model attempt is one generation; a paid response also creates one ledger row. Tool calls are child spans with timing and error status. Strict schemas allow only counts, timing, stop reasons, and tool names—never prompts, answers, arguments, or results. See ReplyUsageMetaSchema and schemas/sandbox-ai.ts.

Wiring: www’s hooks.server.ts and the backstage’s index.ts each call configureUsage({supabase, posthogKey, posthogHost}) once at boot — the ledger needs a service-role client (usage_events RLS rejects request-scoped ones) and core can’t read $env. Scripts call it themselves; unconfigured, recordUsage warns and drops the row. The www hooks also hand flushUsage() to waitUntil after every request so workerd doesn’t cancel in-flight writes; the backstage has no request hook, so every in-process turn and every failure narration drains its own in a finally.

Paid director attempts are metered at the response, before strict parsing can reject a malformed, ineligible, or empty choice. Each billable response records one usage event through packages/core/thread-director/thread-director-decision-usage.ts; a call with no response, tokens, or cost writes no row and surfaces a bounded failure class.

Key-source semantics: llm-keys. Smoke probe: bunx varlock run -- bun run packages/core/scripts/test-llm-tracking.ts — one real LLM call, poll-asserts the usage_events ledger row (sink 1) and prints its trace id; sink 2 (PostHog) is confirmed in the AI Observability dashboards or with posthog-cli exp query run against that trace id (Querying).

Querying

posthog-cli is authenticated (~/.posthog/credentials.json: https://eu.posthog.com, project 133397). Two surfaces:

posthog-cli exp query run "<HogQL>" — the workhorse. HogQL is ClickHouse SQL over an events table. Prints JSON-lines arrays, no column names — pipe through jq/python3 when you need keys, or just remember your own select order. --debug prints the raw envelope. exp query check "<HogQL>" resolves field names against the live schema without running the query; use it before anything long.
posthog-cli api <cmd> — PostHog’s MCP tool catalog as a shell: search <regex>, info <tool>, call <tool> '<json>'. Use it for things HogQL can’t do: insight-create / dashboard-create to persist a query as a tile, docs-search, query-llm-traces-list. Destructive tools require --confirm; preview with --dry-run. posthog-cli api --agent-help is the full guide.

Shell gotcha: every PostHog-native property starts with $, which fish and bash expand inside double quotes. The SQL’s own string literals need the single quotes, so the working shape is double-quoted SQL with each property escaped: "select properties.\$ai_model from events where properties.seam='director'".

What’s in there

family	events	key properties
lifecycle (`track()`, normally opt-in)	`house.created` (always-on), `thread.created`, `user.signed_in`, …	`route`, `method`, `request_id`, `caller_kind`, `release_sha`, `ts`
errors	`server.error` (always-on), `client.error` (opt-in)	`runtime`, `source`/`mechanism`, `code`/`error_kind`, `message_hash`, `route`, `status`, `release_sha`, `fatal`
dispatch mirror (always-on)	`dispatch.started`/`.completed`/`.skipped`/`.attempt_failed`/`.failed`/`.phase`	`thread_id`, `agent_id`, `trace_id`, `entry_id`, `task_id`, `attempt`, `max_attempts`, `duration_ms`, `http_status`, `reason`, `phase`, `round`
usage (always-on)	`$ai_generation` (LLM), `$ai_span` (tools), `arbe_usage` (sandbox/files)	`seam`, `key_source`, `house_id`, `agent_id`, `thread_id`, `trace_id`, `$ai_session_id`, `$ai_trace_id`, `$ai_span_id`, `$ai_parent_id`, `$ai_model`, `$ai_tools_called`, `$ai_tool_call_count`, token/cost/latency fields, plus the bounded director projection (`decision_kind`, `validation_outcome`, `stop_class`, `output_mode`, `candidate_count`, `configured_thinking_level`, `reasoning_tokens`, `cost_source`)

$ai_generation_summary and $ai_trace_summary are PostHog-derived rollups, not ours — don’t sum cost from them and from $ai_generation in the same query.

Join keys. trace_id (mirrored as $ai_trace_id) is the turn: it ties the director call, the reply call, every tool seam, and the dispatch terminals together, and it is the same id as the usage_events.trace_id ledger row. house_id is also on $group_0. distinct_id is the raw agent uuid (falling back to thread or house id), not a person — there is no useful person model, and the groups table is empty because we set $groups on events but never call groupIdentify. Filter houses with $group_0, not by joining groups.

Recipes

# Is data arriving at all, and what kinds?
posthog-cli exp query run "select event, count() as c, max(timestamp) as last
  from events where timestamp > now() - interval 7 day group by event order by c desc"

# Where the LLM money goes, by seam. (30d: reply $1.36, director $1.21, gate $0.64)
posthog-cli exp query run "select properties.seam as seam, count() as calls,
  round(sum(properties.\$ai_total_cost_usd), 4) as usd
  from events where event='\$ai_generation' and timestamp > now() - interval 30 day
  group by seam order by usd desc"

# Director cost per model, with latency.
posthog-cli exp query run "select properties.\$ai_model as model, count() as calls,
  round(sum(properties.\$ai_total_cost_usd), 4) as usd, round(avg(properties.\$ai_latency), 2) as lat
  from events where event='\$ai_generation' and properties.seam='director'
    and timestamp > now() - interval 7 day group by model order by usd desc"

# Are director decisions landing? (needs the meta projection — see caveats)
posthog-cli exp query run "select properties.decision_kind as kind,
  properties.validation_outcome as outcome, count() as n
  from events where event='\$ai_generation' and properties.seam='director'
    and timestamp > now() - interval 7 day group by kind, outcome order by n desc"

# Spend per house, or per turn.
posthog-cli exp query run "select \$group_0 as house, count() as calls,
  round(sum(properties.\$ai_total_cost_usd), 4) as usd
  from events where event='\$ai_generation' and timestamp > now() - interval 7 day
  group by house order by usd desc limit 20"

# One turn, end to end.
posthog-cli exp query run "select timestamp, event, properties.seam, properties.\$ai_model,
  properties.\$ai_total_cost_usd, properties.duration_ms
  from events where properties.trace_id='<trace-id>' order by timestamp"

# Why dispatches don't run. `reason` mixes enums (no_mode, gate_no, filtered,
# debounced, no_targets) with raw provider errors.
posthog-cli exp query run "select properties.reason as reason, count() as n
  from events where event in ('dispatch.failed','dispatch.skipped')
    and timestamp > now() - interval 30 day group by reason order by n desc limit 20"
# Errors by surface and stable message group.
posthog-cli exp query run "select event, properties.runtime, properties.source,
  properties.code, properties.message_hash, count() as n
  from events where event in ('server.error','client.error')
    and timestamp > now() - interval 7 day
  group by event, properties.runtime, properties.source, properties.code,
    properties.message_hash order by n desc limit 50"

A trace_id from any of these opens directly in the AI Observability UI, which renders the turn as a span tree: https://eu.posthog.com/project/133397/ai-observability/traces/<trace-id>.

Caveats

Money lives in the ledger. usage_events (Postgres) is the billing source of truth; PostHog is the exploration surface. Both are best-effort (no outbox — see above), so a PostHog total is a lower bound, and the two can disagree. Reconcile through trace_id.
Token sums undercount. $ai_cache_reporting_exclusive is true: $ai_input_tokens excludes cached reads, which land in $ai_cache_read_input_tokens / $ai_cache_creation_input_tokens. Aggregate $ai_total_cost_usd, not tokens. Some providers report no $ai_latency at all.
seam is an open string, and PostHog keeps retired values forever — gate still holds ~$0.64 of history but stopped being emitted on 2026-08-15. Retired event names linger the same way: house.file.uploaded appears in the event list but nothing emits it since 2026-08-26. Always window your queries by time.
release_sha is not on everything, and its shape varies. Only track() events and backstage server.error carry it, as either a full 40-char sha, a short 7-char one, dev, or null depending on where the process got it. Don’t group by it across event families.
dispatch.completed fires twice per turn, from dispatch.ts (per-bot: has agent_id, trace_id, phases) and from reply-turn.ts (outer backstage task: neither). Count turns with uniq(properties.entry_id), not count().
The director meta projection is new — it started landing 2026-08-24, so decision_kind / validation_outcome are null on everything before that. A poisoned or unknown meta bag also projects nothing rather than partially (see Usage), so null means “not projected”, never “no decision”.
Lifecycle events are consent-gated (alpha default: on — see above); usage and dispatch events are not. Never read a track() event count as a population count.
$ip is still attached to every server event even though $geoip_disable suppresses enrichment.

Code

One server-side PostHog transport: capturePosthogEvent in @arbe/core/posthog (direct POST to /capture/ — workerd-safe; don’t add posthog-node). Both apps/www/src/lib/server/posthog.ts (route capture) and packages/core/usage-posthog.ts (usage sink) delegate to it.

See debugging, observability, llm-keys.

Gaps: signed_up vs signed_in not distinguished (ensureHumanAgent is idempotent); signal.agent.created and signal.member.joined aren’t in core’s vocabulary yet — currently PostHog-only; read-path activity not instrumented; the CLI emits nothing (CLI sign-in never fires user.signed_in); coding boxes, delegate_task and run_command record money and traces but no countable lifecycle; the backstage reports only failures, never a healthy heartbeat. The full list and the pick-or-reject call: task arbe-e9b7.

Rejected: per-entry thread.entry.created PostHog events — too high-volume, weak dashboard value, messy consent; if activity trends are ever needed in PostHog, emit periodic aggregates instead.