Operating

Health, metrics and traces

Four ways to see what a server is doing, from the cheapest to the most detailed: /healthz says what is listening, /v1/stats and /metrics say how it is going, traces say where one request spent its time, and the audit record says who asked for what.

/healthz — what is listening

curl -s localhost:11435/healthz

{"backend":"cpu","id":"590bc10b059c7189","instance":"84d9eee5236ec24f","model":"qwen2.5:0.5b","status":"ok","transcription":"whisper-tiny.en","version":"0.3.0"}

Field	Meaning
`status`	Always `ok` when it answers.
`model`	The default model, as a name a request can send.
`id`	An identifier of the default model's weights.
`instance`	This process. It changes on every restart, so a client can tell a restart from the same server.
`version`	The llamay build.
`backend`	`cpu`, or `device` when a GPU runs the model.
`transcription`	Present when `-whisper` loaded a speech model.
`warning`	Present when the GPU was asked for and bypassed, with the reason.
`models`	`0` on a server that started with an empty store and is waiting for a first pull.

/healthz is the one route that never asks for a key, because what reads it is usually a probe with no way to carry one. It says what is listening, not how busy it is. On a server with -tls-client-ca, the probe still needs a client certificate: the handshake comes first.

/v1/stats — the numbers as JSON

{"arch":"qwen2","backend":"cpu","context":32768,"kv_allocated":5,"kv_bytes":7864320,"kv_freed":0,"kv_pages":5,
 "layers":24,"model":"Qwen2.5 0.5B Instruct","model_id":"590bc10b059c7189","prefix_entries":5,"served":5,
 "sessions":0,"threads":7,"uptime_s":485,"weight_bytes":391859712}

?model= asks about another loaded model. With -batch, a batching object adds the scheduler's counters: active, queued, admitted, completed, rejected, steps and tokens.

/metrics — Prometheus

llamay_up{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 1
llamay_uptime_seconds{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 485
llamay_requests_total{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 5
llamay_weight_bytes{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 391859712
llamay_context_length{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 32768
llamay_kv_pages{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 5
llamay_kv_bytes{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 7864320
llamay_prefix_entries{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 5
llamay_sessions{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 0
llamay_models_resident{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 1
llamay_models_available{model="Qwen2.5 0.5B Instruct",arch="qwen2"} 9
llamay_embeddings_total{model="all-MiniLM-L6-v2"} 1

The model label is the name inside the file, not the name you pulled it as. /metrics reports the default model only, and needs a key like every route but /healthz.

Metric family	Present	Watch it for
`llamay_up`, `uptime_seconds`, `requests_total`	Always	Restarts; traffic
`kv_pages`, `kv_bytes`, `kv_pages_allocated_total`, `kv_pages_freed_total`	Always	Cache memory. Allocated minus freed that only grows means contexts are not being released.
`prefix_entries`, `sessions`	Always	How many shared prefixes and named contexts are held
`batch_active`, `batch_queued`, `batch_rejected_total`, `batch_decode_tokens_total`, `batch_prefill_tokens_total`…	With `-batch`	Queue depth, and 429s: `batch_rejected_total` rising means callers are being turned away
`kv_budget_bytes`, `contexts_spilled`, `context_spills_total`, `context_restores_total`, `context_spill_failures_total`…	With `-ctx-mem-budget`	Spill failures, and restores that are frequent enough to hurt
`models_resident`, `models_available`	With a model store	Loading and evicting
`embeddings_total`; `rerank_requests_total`, `rerank_pairs_total`	With `-embed`; `-rerank`	Retrieval traffic

Traces

llamay serve -otlp http://collector:4318      # or OTEL_EXPORTER_OTLP_ENDPOINT

Each request is a span named for its route, with children for the three phases that decide its latency: queue (waiting for a slot), prefill (reading the prompt) and decode (writing the answer). A caller's traceparent header is joined, so llamay's spans sit inside your application's trace. They are sent as OTLP over HTTP to /v1/traces on the collector.

The split is the point. A request that spent most of a second in queue needs more capacity or -concurrency; one that spent it in decode needs a smaller model or a GPU. Spans carry token counts and stop reasons, never the text. -traces without -otlp writes the spans to the log at debug level.

Logs

-v logs every request: method, route, status and time.
The Linux service logs to the journal: journalctl -u llamay -f.
The Mac app writes ~/Library/Logs/llamay.log; the Windows window writes to %LOCALAPPDATA%\llamay\logs.
A server started by llamay app logs to server.log in the user cache directory's llamay folder.

The audit record

Metrics count; the audit record names. With -audit, every request is one hash-chained JSON line saying who asked, for which route, with which result. llamay audit who turns it into a table by caller:

$ llamay audit who audit.jsonl
caller               requests  refused     tokens  last seen
visitor                     1        0          0  2026-09-27T04:02:41.892951Z
-                           1        1          0  2026-09-27T04:02:41.902559Z
analyst                     1        0          0  2026-09-27T04:02:41.857946Z

- is a request that named no caller — here, one refused with a 401. The format and verification are on deploying for a team.