Skip to content
View as .md

Measure website performance

Start with apps/www/scripts/profile-routes.ts --help. It measures document navigation or client-side thread switching, keeps every sample, and reports medians and ranges. Runs are read-only: opening a thread never submits a message.

Terminal window
cd apps/www
bun run scripts/profile-routes.ts --require-ready --revision before --json \
  /houses /threads/<id> > /tmp/arbe-before.json
# Make one change, let hot reload finish, then repeat the identical command with
# --revision after and output /tmp/arbe-after.json.
bun run scripts/compare-performance.ts /tmp/arbe-before.json /tmp/arbe-after.json


# First visit and revisit from another thread in the same house:
bun run scripts/profile-routes.ts --require-ready --revision before --json \
  --switch-from /threads/<source> /threads/<target> > /tmp/arbe-switch-before.json

The local default is https://arbe.localhost with the arbe-dev auth profile. For a production baseline, pass --origin https://arbe.0sk.ar --profile arbe-prod; readiness measurements require a deployment containing the instrumentation. Setup and auth recovery are in the browser-testing skill. The profiler always runs browser commands from the repository root, where agent-browser.json selects the session. Manual probes must use that same cwd. The browser command is agent-browser auth login; Arbe’s separate CLI uses arbe login.

For fast production-build comparisons without deployment waits, the Cloudflare adapter supports vite preview. Build with bun run --filter '@arbe/www' build, then run portless arbe-preview bunx vite preview from apps/www on its own approved browser hostname. The package’s preview script instead runs remote Wrangler. Pass --build-dir apps/www/.svelte-kit/output to the profiler from the repository root to detect changes to the served build rather than unrelated edits in the shared checkout. Restart only your preview process when switching saved builds; keep both variants’ output immutable and compare on the same origin.

Use fixed threads the robot account can read; botlab is suitable. Include a short conversation and a longer one, and keep the same signal/tool display settings. Defaults are one discarded warmup and five measured samples. HTTP cache state is uncontrolled; these are document navigations in an existing browser session, not cold-start claims. Each switch sample starts with a new source document, visits the target, returns, and revisits; HTTP cache survives while app memory starts fresh. Keep browser, viewport, network conditions, analytics consent, and app origin unchanged. Fixture counts detect additions/removals, not edited message text; leave the fixture contents untouched. Local Vite results cannot stand in for production results.

Keep the shared browser and runtime source files idle during a run. Source fingerprints detect concurrent local code changes; they identify the checkout, not the deployed production revision. Supply the measured revision with --revision. Profiler fingerprints prevent comparisons across different measurement drivers. Redirects, missing readiness/paint, changed thread entry/message counts, and unfinished samples fail the run. Electric’s HTTP 409 handle reset is accepted only when a later request to the same shape succeeds; unrecovered resets still fail. Preserve failed output as diagnostic evidence, never as a baseline. The comparison command rejects incompatible reports and recalculates deltas from raw samples; overlapping ranges warrant another run, not a claimed win. For a stronger causal check, restore only your optimization, repeat the baseline, then reapply and repeat.

Read the metrics as separate phases:

  • FCP / LCP describe paint, sampled after the readiness wait.
  • shellReadyMs marks the mounted header and route after DOM update and two animation frames; it does not mean every house collection has loaded.
  • threadReadyMs additionally waits for successful history backfill, ready history/authors queries, and an authenticated composer, then DOM/scroll frames. Revisits can use previously loaded history; this does not promise a fresh server read. Rendered Markdown enhancements can finish later.
  • Switch timings start immediately before the browser clicks a same-origin link and end at the target’s fresh thread-ready mark. A full document reload fails the scenario.
  • settleMs is the last completed loader/API response, a diagnostic only. Shape requests are excluded even when their initial data is needed; it is not readiness or a complete critical path.

The marks live in apps/www/src/lib/performance.ts, the root layout, and Chat.svelte. They stay in the browser’s User Timing timeline. A navigation commit with startObserved: false means the start callback was missed; the profiler uses its own click timestamp, never substitutes the commit time. Save JSON reports with the change and report absolute timings, sample counts, ranges, origin, and the exact behavior changed. No measured improvement means say so and use the trace to choose the next candidate.

The saved browser proof includes reusable botlab fixtures, raw reports, rejected comparisons, and a repeat that exposed a misleading first-run gain.

The local production-build experiments record rejected optimizations and the cache conditions used; none established a repeatable latency gain. Keep raw traces and build snapshots in ignored local storage, and commit compact evidence instead.

Use agent-browser vitals --json for one current-page reading. Reach for agent-browser trace start|stop or profiler start|stop only after the route profiler identifies a route but not the expensive phase.

If local house/thread pages hang with live=true requests pending, confirm the browser is on https://arbe.localhost and run portless doctor. HTTP/1.1 on Vite’s direct :8888 port can exhaust its connection pool on Electric long-polls. See local development.