Reference

Errors

Every status llamay returns, what causes it, and what to do. The messages below are the ones the server actually emits — they are quoted from the code rather than paraphrased, so searching this page for the string you got will find it.

The shape of an error

Errors come back in the shape of whichever API you called, because a client written for that API already knows how to read it.

// OpenAI and llamay-native routes
{"error": {"message": "messages is empty", "type": "Bad Request"}}

// Ollama-compatible routes
{"error": "no model named \"llama3:8b\""}

type is the HTTP status text rather than a private taxonomy, so there is no second vocabulary to learn and nothing to keep in step with the status line.

Whose mistake was it?

400 — the request needs changing

Asking a model to do what it is not

Three of the 400s exist because the alternative is worse than an error. An encoder asked to generate text produces a vector interpreted as tokens, which is fluent and meaningless.

bge-small is an encoder: it produces one vector per input and cannot generate text.
bge-reranker is a cross-encoder: it scores a query against a document and cannot generate text.
this server is serving qwen2.5-7b, a decoder, and has no encoder loaded.

The last one is the common case: /v1/embeddings against a server started with only -m. Start it with -embed as well — one process can hold a decoder, an encoder and a cross-encoder at once, which is what makes a local retrieval stack a single binary.

401 — no key, or the wrong one

this server requires an API key: send Authorization: Bearer <key>

Sent with WWW-Authenticate: Bearer realm="llamay", so a client knows what is missing rather than guessing at the route. Both spellings work: Authorization: Bearer <key> and X-Api-Key: <key>. The comparison is constant-time.

/healthz is the one route that never asks. Whatever reads it is usually a load balancer or a container probe with no way to carry a secret, and it reports nothing a caller could not learn by connecting.

403 — refused on purpose

A CORS refusal has no error body

This is the one refusal you will not find in a response, because there is nothing to find. When -cors does not name the calling origin the preflight is answered 204 with no CORS headers on it, and the browser refuses the real request on its own — your code sees a network error, not a status.

TypeError: Failed to fetch
Access to fetch at 'http://127.0.0.1:11435/v1/chat/completions' from origin
'https://example.com' has been blocked by CORS policy

That message comes from the browser console, not from llamay. Start the server with -cors <your origin>. Saying no by saying nothing is deliberate: it keeps a site that merely guessed out of the logs, and there is no useful thing to tell a caller that was never authorised.

404 — named something that is not there

no model named "llama3:8b". Available:
  qwen2.5-7b-instruct:q4_k_m
  gemma2:2b (ollama)

The list is the point: a miss that only says "not found" leaves you guessing at names you have no way to enumerate. It spans both llamay's store and Ollama's.

no such context on this model: "ctx_9". A context belongs to the model it was
created on; if this one was built against another model, create it again
against this one.

Contexts are per engine and an engine is per model, and ids on two models both start at ctx_1. KV computed under one set of weights is meaningless under another, so this is refused rather than reinterpreted. On the context routes the model is a query parameter: POST /v1/contexts/ctx_1/fork?model=qwen2.5-7b.

413 — larger than this server takes

A body limit, checked before the bytes are read rather than after. A file the platform cannot carry is not a quota problem and must not be described as one.

429 — the queue is full

HTTP/1.1 429 Too Many Requests
Retry-After: 1

Requests past -max-queue are refused rather than queued. A request that will wait longer than the client will wait is work nobody reads, so shedding it is cheaper than serving it — and a 429 with a Retry-After is something a client can act on, which a 500 is not.

If you see these steadily rather than in bursts, the fix is capacity or admission control, not a shorter backoff: raise -conc if the device has headroom, add an instance if it does not, and put a queue in front — see deployment patterns.

503 — nothing is loaded yet

this server has no model. Pull one:
  POST /api/pull {"model": "..."}

A generation route on a server whose store is empty. It names the route that ends the state rather than only reporting it, and /api/pull is served in this state on purpose — it is the only way out.

500 — ours

A genuine server fault: a file that will not read, a device that will not allocate, a bug. The message is the underlying error rather than a generic one. If you can reproduce it, the message and the llamay version output are the two things worth including in a report.

One class of 500 is worth knowing about because it used to be miscategorised: an unknown model name on the context routes answered 500 while every chat route answered 404 for the identical error. That is fixed — a name with a typo in it is the request's mistake, and a 500 tells the client nothing it can act on while paging somebody who cannot help.

Errors during a stream

A stream that has already begun has sent 200 and its headers, so a later failure cannot change the status. The connection ends without a [DONE] frame.

saw_done = False
for line in stream:
    if line.strip() == "data: [DONE]":
        saw_done = True

if not saw_done:
    # Truncated. Do not treat what you have as a complete answer.
    raise RuntimeError("the stream ended early")

Checking for [DONE] is the only way to tell a finished answer from a truncated one, and it is the check most clients skip.

Not errors: finish reasons

A 200 with a short answer is not a failure, and the reason says which kind of ending it was.

On the Anthropic route these are end_turn, stop_sequence, max_tokens and tool_use, and stop_sequence is reported separately from end_turn because a caller's delimiter arriving is a different event from the model finishing its sentence.

Where to go next

The HTTP API documents every field these errors are about. Serving covers -max-queue, -conc and what is logged. Install has the failures that happen before the server starts.