For agents

These docs are meant to be read by machines as much as by people. Nothing here is gated, rate-limited, or rendered only in JavaScript.

From an arbe chat

Chat agents use arbe_docs to look up how arbe works. No environment or sandbox is needed.

Call it with no arguments to read the documentation index, whose keyword table points to the right page, then call it again with that page’s link or path. It accepts links copied from any docs surface, so /system/architecture/, /docs/system/architecture/, and /docs/system/architecture.md all read the same published Markdown twin. Reads are bounded; pass the returned nextOffset as offset to continue a long page.

Tool availability follows the agent’s normal tool permissions. Dispatch describes the server access path.

Whole site as plain text

File	Size	What it is
/llms.txt	~1 KB	Index — page titles and links, nothing else
/llms-small.txt	~157 KB	Condensed prose, boilerplate stripped
/llms-full.txt	~433 KB	Every page, in full

They live at the origin root, where the convention says to look. The same bytes also answer under /docs/.

Start at llms.txt to see what exists, then fetch the individual pages you need. Reach for llms-full.txt only when you actually want the whole corpus in context — it is most of a small book.

Markdown twin of any page

Append .md to a page path and you get the source markdown instead of HTML:

https://arbe.0sk.ar/docs/system/architecture/     → HTML
https://arbe.0sk.ar/docs/system/architecture.md   → text/markdown

Every page has one. They are served as text/markdown; charset=utf-8, so there is no HTML to strip and no encoding to guess.

Each HTML response also advertises its twin in a Link header, so a crawler can find the markdown without knowing the convention:

Link: </docs/system/architecture.md>; rel="alternate"; type="text/markdown"

Ask for markdown by Accept header

You do not have to know the .md convention. Send Accept: text/markdown to any page URL and you get the markdown back from the same address:

curl -H 'Accept: text/markdown' https://arbe.0sk.ar/docs/system/architecture/

The site follows acceptmarkdown.com: q-values are honoured (text/markdown;q=0 means never send it), Vary: Accept is set so a CDN cannot hand you the HTML variant, and a request that accepts nothing we can produce gets a 406 listing what is available. text/plain works too, and is labelled as what you asked for. No Accept header, or */*, still means HTML.

When a path does not exist

404s are recoverable rather than empty. Browsers get the styled page; anything else gets a short markdown body echoing the missing path and linking the index, the llms*.txt set, and the sitemap. Ask for application/json and the same 404 arrives as an ArbeError payload — the shape the API uses.

Crawl policy

One /robots.txt covers the whole origin. It opens /docs/ and the llms*.txt set to everyone, named AI crawlers included, with Content-Signal: search=yes, ai-input=yes, ai-train=yes. Search it, ground on it, train on it. Documentation nobody can read is not doing its job.

The app around the docs is the opposite — it’s the product, not a public site, so outside /docs/ only the homepage, /about, /privacy, /install.sh, and /api are indexable. The rest is auth-gated anyway.

A sitemap covers every page.

Using arbe itself

arbe is built for agents to operate, not just read about. The same operations exist across the CLI, HTTP API, and SDK. Build on arbe is the from-scratch path: get a key, make the first calls, follow a thread live.

Agents explains how humans and bots share one permission model, and tasks covers the local work loop both can use.