Reference

Troubleshooting and FAQ

The things developers hit, roughly in the order they hit them, each with what causes it and what to do. For every status code and message, see errors.

"address already in use"

Something already holds 11435 — usually another llamay: the desktop app, the Linux service, or a llamay app you started earlier. Ask it who it is before stopping anything: 

curl -s localhost:11435/healthz          # model, version, instance
lsof -iTCP:11435 -sTCP:LISTEN            # macOS and Linux: which process
llamay serve -addr localhost:11436       # or run beside it

The wrong model answered

401 with a key that works elsewhere

"uses X, which llamay's constraint cannot honour"

The schema has a keyword that cannot be enforced — not, if/then, patternProperties, a remote $ref. llamay refuses rather than silently not enforcing it. Remove the keyword, or restate it with what is supported; the support matrix lists every keyword.

The JSON is valid but the values are wrong

A schema fixes the shape: keys, types, enums, bounds, patterns. The values are still the model's. A 0.5B model asked for one order line may write three, all valid. Use a bigger model, give an example in the prompt, and keep checking values in your code.

The model answered in prose instead of calling the tool

Under tool_choice: "auto" the model decides, and small models often decline. Send "required" (or a named function) and the call is constrained to the tool's schema, so it cannot come back as prose. Several calls in one turn are allowed unless parallel_tool_calls is false; how many is still the model's choice.

"no such context" after a while

A context saves nothing: cached_tokens is near zero

A request continuing a context must start with exactly what the context holds. If you built it from a system message, send that same system message first in every request, then the new turn. Where the prompt stops matching, the context is cut back to the common part and the rest is computed again — correct, but not cheap. Check usage.prompt_tokens_details.cached_tokens; see using a context.

409 "this context is in use by another request"

One request writes to a context at a time. For several at once, fork it: POST /v1/contexts/{id}/fork?n=8 gives eight contexts that share its pages, and each can be used in parallel.

429 "server is at capacity"

The queue holds 128 waiting requests; past that, the server says no at once with Retry-After: 1. Back off and retry. If it is steady rather than bursty, raise -concurrency if the machine has headroom, turn on -batch, or add a server. Transcription allows four at a time and answers 429 past that.

It is slower than expected

  1. /healthz: is backend device or cpu? The plain macOS tarball has no Metal; the Linux GPU build is _gpu. llamay version says what is built in.
  2. The quantisation: a q8_0 reads twice the memory of q4_k_m per token.
  3. Requests with logprobs, images or LoRA adapters take slower paths — one at a time, or not on the device graph.
  4. The prompt: a long one that changes at the start every time (a timestamp in the system message) is never reused from the cache.
  5. Measure with llamay bench rather than by feel. See throughput.

An image request is refused, or takes minutes

Images are read on the CPU path only in this version: serve with -gpu=false, or a GPU server answers 400. On a laptop CPU a 4B vision model takes one to two minutes per image. See images, scans and video.

415 "not a WAV file"

Transcription reads WAV only. Convert first: ffmpeg -i talk.m4a -ar 16000 -ac 1 talk.wav. See speech to text.

"Failed to fetch" from a web page

The browser refused, not llamay: the server sends no CORS headers unless -cors names the page's origin. Start it with -cors https://your.app.example. There is no wildcard.

Studio answers 421 from another machine

Studio answers only requests addressed to localhost, a loopback address or a .local name, so a server on a network does not hand Studio to the network. Use it on the machine running the server, or through an SSH tunnel: ssh -L 11435:localhost:11435 host.

A stream stopped without [DONE]

A failure after streaming began cannot change the status, which was already 200. The stream ends without its data: [DONE] line. Treat a stream without it as truncated.

403 "names its weights by layout only"

A snapshot made by an older llamay records its model's layout but not the SHA-256 of its weights. Under -profile enterprise, or on a request with a classification marking, that is refused. Take the snapshot again from a 0.3.0 server, or start the receiving server with -allow-legacy-snapshots if you accept the weaker check.

"no llamay store"

There is no home directory to put ~/.llamay/models in — common under systemd and in containers. Set LLAMAY_MODELS to a directory the process can write.

Stop the update check

LLAMAY_NO_UPDATE_CHECK=1 in the environment of the command line, the server, and the Windows and Linux windows. The Mac app has Check for Updates Automatically in its menu.

Still stuck

Open an issue on the public issue tracker with the command, the full message, and the output of llamay version. Security problems go to [email protected] instead.