Every command llamay has and every flow worth seeing end to end, recorded as
it ran. Nothing here is a mock-up: each one is a pseudo-terminal, a real
binary and the output it actually produced, on a machine this page names.
How these were made, and what that is worth
A demo is a claim about what a program does, and a claim is only worth what
produced it. These were produced by
scripts/site/record_demos.py, which runs the command you are
reading and records the bytes that came back, at the times they came back.
Re-running it re-records all of them.
Three things in each recording are synthesised and everything else is real,
which is the same disclosure the script carries at the top of its own file:
The prompt.~/llamay $ is not anybody's shell. A
recording of a real one is a recording of its theme, its git branch and
its aliases, none of which are llamay.
The typing. Characters arrive at a human-ish rate because a command
that appears all at once reads as a screenshot. The string being typed is
the string that ran.
The comment lines. They are captions wearing a #.
The output is not touched. It is not re-ordered, not trimmed to look faster,
and not typed back in afterwards. Where a command failed it was recorded
failing — a snapshot refusing the wrong
weights, a request without a key and
four errors in a row are demos here for the
same reason the throughput page has a column llamay loses: the refusal path
is the one people deploy without checking.
The numbers are this laptop's. Each player prints the machine, the
build and the date underneath it. An Apple M4 running the Metal build is not
your server, and a demo is not a benchmark —
the throughput page is where the measured
comparisons live, with the method written down.
The player
These are recordings, not video. Each one is an asciicast — a
few kilobytes of JSON — replayed by a terminal emulator in
/assets/cast.js that llamay.com ships itself. That choice buys
three things a <video> tag cannot:
The text is text. Select it, copy it, search it with the browser's
own find, let a screen reader walk it. A demo of a command-line tool whose
output you cannot copy is a poster of a command-line tool.
It is small. Thirty-five sessions come to about four hundred
kilobytes in total, fetched only when you scroll to one. The equivalent in
MP4 is a hundred times that.
It diffs. A recording is a text file in the same repository as the
code it records, so a command whose output changed shows up in a diff
rather than in a screenshot somebody has to notice is stale.
Space plays and pauses, the arrow keys seek five seconds, 0
returns to the start, and copy puts the session's commands — without
the prompts or the output — on your clipboard. Below each player is the
whole session as plain text, which is what the page is when JavaScript does
not run.
Install to first token. One binary, one model already in the store, one command. Nothing is configured and nothing is sent anywhere.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:29
The one-line install, checksum and all. curl | sh, from a real release: it works out the platform, fetches SHA256SUMS, refuses the binary if the sum does not match, and installs. Here it installs into a scratch directory instead of /usr/local/bin.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:11
Pulling a model. Ollama-style names, Hugging Face repositories and plain URLs all land in the same store. Transfers resume; this one is forced to start over so there is something to watch.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 1:02
Streaming, and what it cost. The default: continue the prompt, stream every token as it is sampled, and print prefill and decode rates when it stops.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:17
Chat templates, applied from the file. -chat renders the prompt through whichever template the GGUF declares, so the same command is correct for Gemma, Qwen and Llama without a flag per family.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:21
JSON that is JSON. -json puts a grammar over the sampler. Invalid tokens are not merely unlikely, they are unreachable, so the output parses on the first try from a model far too small to be trusted to remember a brace.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:28
GBNF: a shape you wrote yourself. When the shape is not JSON, describe it. The grammar is compiled to a token mask and applied at every step.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:20
The same seed is the same text. Twice at temperature 0.7 with one seed, then the same seed on one thread and on eight. Determinism here is a property of the scheduler, not of greedy sampling.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:27
What a quantised KV cache costs. f32, then vq8 — values quantised, keys left exact. The cache is a quarter of the size; the demo is whether the text changes.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:21
Drafting from the prompt itself, and when it does not pay. -lookup proposes whatever followed this suffix earlier in the context and verifies it against the model. Where the continuation repeats the context — an edit, a quotation, a diff — almost every draft is kept. Where it does not, the drafts are rejected and it is a rounding error slower. Both are here.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:47
Reading a GGUF before you trust it. Architecture, tensor formats and their counts, the tokenizer, the context length the file claims. This is the command that catches a file whose name says Q4_K_M and whose tensors say otherwise.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:04
Every tensor and every metadata key. -tensors and -meta print the file rather than a summary of it, and -tokenize proves a round trip on your own text.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:08
The store: list, alias, remove. Models are content-addressed blobs with names pointing at them, so a second name costs nothing and removing one never removes weights another name still uses.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:11
Requantising a model, and what it costs. A file in, a smaller file out, and the two measured against the same text so the loss is a number rather than a shrug.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:11
An importance matrix, then a better quantisation. Measure which columns the model actually drives on text you care about, then let the scale search weight them. The same target format, fitted to your corpus rather than to a generic one.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:12
Perplexity, as a way to compare two files. The standard check that a quantisation did not break the model, run over a text file you choose rather than a benchmark someone else chose.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:15
Forking a context eight ways. The thing llama.cpp has no name for. One prefill, eight branches, copy-on-write pages — the copy that was not made is the number to watch.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:15
Save a context, close the laptop, carry on. A prefilled context written to a file, restored into a fresh process, and generation continuing from it — no re-reading of the prompt.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:15
A snapshot that refuses the wrong weights. A context is only meaningful under the model that produced it. The file records which one, and a restore under different weights is an error rather than quiet nonsense.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:10
Contexts over HTTP. The same object through the API: create one, fork it, snapshot it to bytes, restore it. This is how an agent fleet shares one prefill.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:21
An OpenAI endpoint on your own machine. llamay serve, then the request every client already sends — non-streaming, then streaming, with the base URL as the only thing that changed.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:29
Three APIs, one server, one model. OpenAI, Anthropic and Ollama all answer on the same port against the same loaded weights. Nothing is translated at a proxy; the routes are native.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:27
Eight requests at once. Continuous batching: eight generations enter the scheduler, decode together, and leave when each one is done rather than when the slowest is.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:20
A key, and what happens without one. -api-key closes every route but /healthz. The demo is the 401, because that is the one people deploy without checking.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:19
Tool calls that fit the tool's own schema. The tool's JSON Schema becomes the grammar over the sampler, so the arguments the model returns are arguments your function can take.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:22
Embeddings beside the decoder. One process, one port: an encoder served at /v1/embeddings next to the chat model, so a retrieval pipeline is local end to end.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:17
Similarity, without leaving the machine. llamay embed -sim prints the matrix, which is the only thing most retrieval work needs and the thing an API bill is usually for.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:08
The invariants, checked on your own file. Eight checks, including one that is expected to fail and fails: a deliberately destroyed quantiser, there so the bound above it is not vacuous.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:05
The same checks on a real model. The tiny file in testdata is there so the check is fast. It is the same command on weights you would actually serve.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:04
Prefill and decode, measured. The command behind the table on the throughput page. It reports the two phases apart because they are bound by different things.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:09
What the state engine is worth, at 2048 tokens. The same benchmark with a prompt worth keeping and thirty-two branches off it. The line to read is the last one: restoring that context against computing it again.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:10
Being told no, four ways. Every status llamay returns is emitted by a line you can read. These are four of them, produced rather than quoted.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:21
Changing one line in an existing client. A script written against OpenAI, pointed at llamay by its base URL. The request, the response shape and the streaming format are the ones the client already expects.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:26
Extraction under a schema, end to end. A page of unstructured text in, a record that validates out — the shortest useful thing to build on a local model, complete. The schema is a file you can read, not a wall of escaped quotes.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:24
Pulling the plug. The claim is that llamay works with no network. This is that claim, tested: the loopback interface only, no DNS, a full generation.Recorded 2026-09-18 · Apple M4, macOS 26.0.1 · llamay v0.1.58-1-gb14f550 · metal:Apple M4 · 0:19
Running these yourself
Every one of them is a command you can paste. The models are ordinary
downloads —
llamay pull qwen2.5:0.5b, llamay pull gemma2:2b,
llamay pull all-minilm:latest — and the files under
testdata/ ship with the source. The recording script itself is
the last word on what each demo does:
make demos
# what there is
python3 scripts/site/record_demos.py --list
# re-record one of them
make demos DEMO=ctx-fork
# re-record all of them, which takes about a quarter of an hour
make demos
The casts are committed, so the site never depends on a recording machine
being available at deploy time. make site-check proves the HTML
on disk is what the generator emits from them; make demo-check
proves every demo a page references has a recording and that none of them
recorded a failure.