Capabilities

Documents, search and citations

Point llamay at a folder of documents and it builds an index the server can search. Every result carries the file it came from and the exact byte range, so an answer built on it can cite a place anybody can open. Labels decide which caller may see which passage, before anything is ranked.

The filter runs before ranking, so a passage a caller may not see never appears as a result, a score or a count.

1 · Build an index

llamay pull all-minilm
llamay index build -embed all-minilm -o manuals.idx -labels labels.txt ./manuals

indexed 3 passages from 3 documents into manuals.idx

What it reads: .txt, .md, .markdown, .rst, .csv, .tsv, .json, .jsonl, .log, .xml, .yaml, .html and Word .docx.
What it skips, by name: everything else, PDFs included. Convert a PDF with pdftotext first; a scanned one can be read with the vision path.
Passages fit the encoder: cut between paragraphs where possible, then sentences, then words. -chunk sets the size in tokens.
The index records its encoder. A server started with a different -embed refuses to load it, because vectors from two encoders are not comparable.

2 · Serve it

llamay serve -m gemma2:2b -embed all-minilm -index manuals.idx \
  -index-access access.txt -api-keys keys.txt

Then ask it, as a named caller:

curl -s localhost:11435/v1/retrieve -H "Authorization: Bearer $LLAMAY_API_KEY" \
  -d '{"query": "why did the pump seal fail", "top_k": 2}'

{"data":[{"source":"restricted/incident.md","start":0,"end":145,
  "text":"# Incident 2026-07\n\nThe seal on pump 3 failed at 410 hours after an unapproved grease was used. ...",
  "labels":["reliability"],"score":0.7389614},
 {"source":"public/pump.md","start":0,"end":200,
  "text":"# Pump maintenance\n\nThe pump seal is inspected every 500 operating hours. ...","score":0.5564259}],
 "index_model":"03a47eb83b925050","model":"all-MiniLM-L6-v2","object":"list"}

Field	Meaning
`query`	Required. The question.
`top_k`	Passages to return. Default 5, at most 50.
`rerank`	With `-rerank` loaded, the encoder gathers four times `top_k` candidates and the cross-encoder orders them. On by default when a reranker is loaded; `false` turns it off.

Each result has source, start and end (byte offsets into the original file), text, its labels if any, score, and rerank_score when a reranker ordered it.

3 · Answer with citations

/v1/retrieve finds passages; it does not write the answer. Hand the passages to a model, numbered, and keep where each came from:

question = "How often is the pump seal inspected?"
hits = post("/v1/retrieve", {"query": question, "top_k": 2})["data"]

sources = "\n\n".join("[%d] %s" % (i + 1, h["text"].strip()) for i, h in enumerate(hits))
answer = post("/v1/chat/completions", {
    "model": "gemma2:2b", "temperature": 0, "max_tokens": 60,
    "messages": [
        {"role": "system", "content":
         "Answer only from the numbered passages. Cite the passage number in brackets."},
        {"role": "user", "content": sources + "\n\nQuestion: " + question},
    ],
})["choices"][0]["message"]["content"]

print(answer)
for i, h in enumerate(hits):
    print("[%d] %s, bytes %d-%d" % (i + 1, h["source"], h["start"], h["end"]))

[1] Every 500 operating hours.

[1] public/pump.md, bytes 0-200
[2] public/valves.txt, bytes 0-115

post is a four-line helper around urllib.request that sends the key; the whole script is in the recipes. This ran as the visitor caller, which is why the restricted incident report is not among the passages.

Who may see what

Two small files. The labels file, given to index build, tags documents by path. The access file, given to serve, grants callers labels.

# labels.txt — pattern: label[, label]
restricted/**: reliability
hr/*.docx: personnel, confidential

# access.txt — caller: label, ...
analyst: reliability
hr-team: personnel, confidential
*: public

A caller sees a passage only if they hold every label on it. Unlabelled passages are visible to every caller.
Patterns match the path relative to what was indexed. * does not cross a /; a trailing /** matches the directory and everything under it.
* in the access file grants a label to everyone. With no access file, only unlabelled passages are visible.
Callers are named however the server names them: a key's id, a smart card's name, or a directory sign-in. See naming callers.
Labels are independent of classification markings: retrieval filters by the caller, not by the request's marking.

From the command line, see what one caller would see:

llamay index query -i manuals.idx -embed all-minilm -k 3 \
  -as visitor -access access.txt "why did the pump seal fail"

Encrypting the index

The index holds the documents' text. index build -key seals it with a 256-bit key file; the server opens it with the same file as -context-key:

openssl rand -hex 32 > index.key
llamay index build -embed all-minilm -key index.key -o manuals.idx ./manuals
llamay serve -m gemma2:2b -embed all-minilm -index manuals.idx -context-key index.key

Limits

An index is built once. Rebuild it when the documents change; there is no incremental update.
Search is by meaning (vectors) and optionally reranking. There is no keyword or hybrid search.
One index per server.