Build with llamay
By industry
What changes when the text you are sending a model is a patient record, a trade, a case file or a classified document. The architecture that matters is the same in every one of these: the model runs where the data already is, and nothing crosses a boundary you did not choose.
The one property that matters
Running with no network at all
llamay has no telemetry and no call home. A machine with the binary and a model file needs nothing else, which is what makes an air-gapped deployment ordinary rather than a special build. A commercial seat licence, where one applies, is a signed file copied across with the binary and checked on the machine — there is no activation step and still nothing to call home to.
# On a connected machine, once:
llamay pull qwen2.5:7b
llamay models path # where the weights landed
# Copy the binary and that directory to the isolated machine, then:
llamay serve # this machine only: the default addressWeights are content-addressed, so a file copied by hand verifies the same way a downloaded one does. Models and formats covers the store layout.
The property, demonstrated:
Transcript — the whole session as text
~/llamay $ # prove there is a network first
~/llamay $ curl -s -o /dev/null -w 'github.com: %{http_code}\n' -m 8 https://github.com
github.com: 200
~/llamay $ # now ask llamay for 80 tokens with the resolver pointed at nothing
~/llamay $ env -i PATH="$PATH" HOME="$HOME" RES_OPTIONS=attempts:0 \
> llamay run -m qwen2.5:0.5b -chat -n 80 -p "Name one thing a local model can do that a ho
sted one cannot."
A local model can do the following:
- Generate and update content faster and more efficiently than a hosted model.
- Improve the performance and efficiency of web applications by caching and optimizing the m
odel.
- Offer a more personalized experience for users by adapting to their specific needs and pre
ferences.
- Offer better accessibility and scalability for local content distribution and delivery.
prompt 22 tokens in 170ms (129.0 tok/s) · generated 68 in 5.168s (13.2 tok/s) · stop: eog
~/llamay $ # no resolver, no token, no telemetry — the weights were already here
Financial services
The work. Summarising filings and research, drafting and reviewing credit memos, extracting terms from contracts, classifying transactions, answering questions over an internal document set.
What llamay changes. Material non-public information, positions and client identifiers never leave the estate, so the model is not a new recipient in your data map. Deployments can be per-desk, which keeps an information barrier a physical fact rather than a policy applied to a shared endpoint.
Practically. Use JSON Schema output for
extraction so a downstream system gets a typed object rather than prose to
parse. Log the model name and version alongside every generated artefact —
llamay version prints a build revision, and reproducing an answer
later means knowing what produced it.
Healthcare and life sciences
The work. Drafting clinical documentation, summarising literature, structuring trial protocols, coding encounters, answering questions over a formulary or a set of standard operating procedures.
What llamay changes. Patient data and identifiable health information stay on the hardware you already control and already assessed. There is no sub-processor to add, no cross-border transfer to justify, and no retention question about a prompt held by somebody else.
Practically. Run one instance per environment rather than sharing one across research and clinical use — separation of duties is easier to demonstrate than to argue. Pin a model version and record it with each output; a regulator asking why a document says what it says is asking about a specific set of weights.
A model that drafts clinical text is a drafting aid and not a decision. Where an output informs care, the clinical review that already exists is the control, and llamay does not change that.
Defence and government
The work. Summarising and searching document sets, drafting on classified networks, translation, structuring reports.
What llamay changes. It is one static binary with no dependencies, no interpreter and no network requirement, which is a shape a security review can actually inspect. Every release is published with checksums signed through Sigstore (verifying a release), and the controls a review asks about — named callers, TLS, an audit record, classification markings — are on the security review page. llamay itself has not been accredited.
Practically. Transfer the binary and the model store through whatever your process already uses for signed artefacts. Nothing has to be reachable from the internet at install time or afterwards.
Security operations
The work. Triaging alerts, summarising incidents, explaining a sample or a log excerpt, drafting detections, answering questions over runbooks.
What llamay changes. Indicators, internal topology and unreleased vulnerability detail are exactly the material an organisation does not want sitting in somebody else's prompt history. Running locally removes that exposure, and it means analysis still works when the incident you are handling is a network one.
Practically. The Ollama-shaped routes mean existing SOAR tooling written against Ollama works without modification. Use a forked context to hold a large runbook once and branch it per alert rather than resending it on every triage.
Legal and professional services
The work. Review, summarisation, clause extraction, drafting from precedent, answering questions over a matter's documents.
What llamay changes. Privilege survives because nothing is disclosed to a third party. For a firm, that removes the awkward question of whether a vendor's terms are compatible with a duty of confidentiality.
The pattern, run end to end on a document that could not have left the building:
Transcript — the whole session as text
~/llamay $ llamay serve -m gemma2:2b -addr localhost:11439
time=2026-09-18T08:51:26.911-04:00 level=INFO msg="device graph attached" max-tokens=256
time=2026-09-18T08:51:26.911-04:00 level=INFO msg="llamay serving" addr=localhost:11439 mode
l=gemma2:2b arch=gemma2 max-loaded=2 keepalive=5m0s params="1.59 GB" context=8192 threads=7
kv=f32 lexicon=0
OpenAI POST http://localhost:11439/v1/chat/completions
Anthropic POST http://localhost:11439/v1/messages
Embeddings POST http://localhost:11439/v1/embeddings (start with -embed <encoder.gguf
>)
Rerank POST http://localhost:11439/rerank (start with -rerank <reranker.gg
uf>)
Contexts POST http://localhost:11439/v1/contexts (then /{id}/fork?n=8, /{id}/sna
pshot)
Models POST http://localhost:11439/api/pull (and /api/copy, /api/delete)
Stats GET http://localhost:11439/v1/stats
~/llamay $ # the server stays up; the rest of this is a second terminal
~/llamay $ # the document, which is the kind that cannot leave the building
~/llamay $ cat build/demo/work/note.txt
Discharge summary. Patient: Anita Sharma, age 62. Admitted 2026-03-04 with chest pain; tropo
nin normal throughout. Discharged 2026-03-06 on aspirin 75mg daily and atorvastatin 20mg nig
htly. Cardiology follow-up in six weeks.
~/llamay $ # the request, with the schema the answer has to fit
~/llamay $ cat scripts/site/demo_extract.json
{
"model": "gemma2:2b",
"max_tokens": 200,
"temperature": 0,
"messages": [{ "role": "user", "content": "replaced by the note" }],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "discharge",
"schema": {
"type": "object",
"properties": {
"patient": { "type": "string" },
"age": { "type": "integer" },
"admitted": { "type": "string" },
"discharged": { "type": "string" },
"medications": { "type": "array", "items": { "type": "string" } },
"followup_weeks": { "type": "integer" }
},
"required": ["patient", "age", "admitted", "discharged",
"medications", "followup_weeks"],
"additionalProperties": false
}
}
}
}
~/llamay $ # put the note in the request, send it, and validate what comes back
~/llamay $ jq --rawfile note build/demo/work/note.txt \
> '.messages[0].content = "Extract the record.\n\n" + $note' \
> scripts/site/demo_extract.json \
> | curl -s http://localhost:11439/v1/chat/completions -H 'content-type: application/json' -
d @- \
> | jq -r '.choices[0].message.content' | jq
{
"patient": "Anita Sharma",
"age": 62,
"admitted": "2026-03-04",
"discharged": "2026-03-06",
"medications": [
"aspirin 75mg daily",
"atorvastatin 20mg nightly"
],
"followup_weeks": 6
}
~/llamay $ # nothing left the machine, which for this text is the requirement
The pattern underneath all of them
Every section above is the same architecture with a different regulator's name attached. If you take one thing from this page, take the shape:
- Run it where the data already is. A model reaching the data is cheaper to assess than data reaching a model.
- Pin the model and record it. An output you cannot attribute to a specific set of weights is an output you cannot defend later.
- Constrain the output. A schema turns a generated answer into a typed object something else can validate.
- Keep the human control you already have. llamay produces drafts. Whatever reviews drafts today should still review these.
Where to go next
- Integrating llamay — the five steps, with code.
- Serving — running it for more than one person.
- Structured output and tools — typed output instead of prose.
- Correctness — what is checked, and what is not.