~/llamay $ llamay serve -m qwen2.5:0.5b -addr 127.0.0.1:11439 -batch 8 -concurrency 8
time=2026-09-18T08:49:16.060-04:00 level=INFO msg="device graph not built: -batch schedules
several sequences per step and the graph runs one context per pass, so serving continues on
the CPU with GPU-offloaded matmuls. Drop -batch to decode on the graph"
time=2026-09-18T08:49:16.061-04:00 level=INFO msg="llamay serving" addr=127.0.0.1:11439 mode
l=qwen2.5:0.5b arch=qwen2 max-loaded=2 keepalive=5m0s params="0.36 GB" context=32768 threads
=7 kv=f32 lexicon=0

  OpenAI      POST http://127.0.0.1:11439/v1/chat/completions
  Anthropic   POST http://127.0.0.1:11439/v1/messages
  Embeddings  POST http://127.0.0.1:11439/v1/embeddings     (start with -embed <encoder.gguf
>)
  Rerank      POST http://127.0.0.1:11439/rerank            (start with -rerank <reranker.gg
uf>)
  Contexts    POST http://127.0.0.1:11439/v1/contexts        (then /{id}/fork?n=8, /{id}/sna
pshot)
  Models      POST http://127.0.0.1:11439/api/pull          (and /api/copy, /api/delete)
  Stats       GET  http://127.0.0.1:11439/v1/stats


~/llamay $ # the server stays up; the rest of this is a second terminal
~/llamay $ time (for i in 1 2 3 4 5 6 7 8; do \
>   curl -s http://127.0.0.1:11439/v1/chat/completions -H 'content-type: application/json' \
>     -d "{\"model\":\"qwen2.5:0.5b\",\"max_tokens\":48,\"messages\":[{\"role\":\"user\",\"c
ontent\":\"Fact number $i about storage:\"}]}" \
>     | jq -r '.choices[0].message.content' & \
> done; wait)
Fact number 5 about storage is: Data stored in a computer system is known as data storage.
Fact number 1 about storage is:

Most of the storage used in today's computers is in the form of RAM (Random Access Memory).
RAM is volatile, which means that when the computer is turned off, the data stored in RAM is
Fact number 3 about storage is:

Storage capacity refers to the total amount of data that can be stored in a given space or o
n a device. It is often measured in terms of storage units, such as gigabytes (GB), ter
Fact number 2 about storage is:

Many types of storage devices, such as hard drives, solid-state drives (SSDs), and solid-sta
te hard drives (SSHDs), are now available in both external and internal forms, allowing for
Fact number 4 about storage is: Storage can be classified into two main types: hard disk dri
ves (HDDs) and solid-state drives (SSDs). HDDs are used in traditional computers and laptops
, while SSDs are used
Fact number 7 about storage is: "Data is stored in various types of physical and digital sto
rage devices, such as hard drives, solid-state drives (SSDs), magnetic tapes, and optical di
scs, as well as in memory and other
Fact number 6 about storage is: Data stored in a database can be accessed and manipulated us
ing various programming languages, such as SQL, Python, and Java. This allows for efficient
data retrieval and modification, and enables users to perform complex operations
Fact number 8 about storage is: Data can be stored in a variety of physical and digital form
s, including but not limited to:

  * Physical storage: Data can be stored in tangible physical locations such as hard drives,
 tape drives,

real    0m22.953s
user    0m0.063s
sys     0m0.077s
~/llamay $ curl -s http://127.0.0.1:11439/v1/stats | jq
{
  "arch": "qwen2",
  "backend": "metal:Apple M4",
  "batching": {
    "active": 0,
    "admitted": 8,
    "batch_avg": 7.270833333333333,
    "completed": 8,
    "decode_steps": 48,
    "decode_tokens": 349,
    "device_bypassed": false,
    "device_prefill": 0,
    "device_rows": 0,
    "prefill_tokens": 120,
    "queued": 0,
    "rejected": 0,
    "steps": 49
  },
  "context": 32768,
  "kv_allocated": 8,
  "kv_bytes": 12582912,
  "kv_freed": 0,
  "kv_pages": 8,
  "layers": 24,
  "model": "Qwen2.5 0.5B Instruct",
  "model_id": "590bc10b059c7189",
  "prefix_entries": 8,
  "served": 8,
  "sessions": 0,
  "threads": 7,
  "uptime_s": 23,
  "weight_bytes": 391859712
}