· News · 7 min read

llamay 0.4.15 adds signed installers and easy server control for local inference

The new llamay 0.4.15 installer is cryptographically signed, offers a one-line install on macOS, Linux and Windows, and lets you start or skip the background API server.

On 29 October 2026 Kosa8 Labs shipped llamay 0.4.15 with signed installers on every platform and a way to install without the background server. Ollama 0.40.13 brought the LFM3 models to Linux, Windows and Apple silicon, and the llama.cpp world had a busy week too.

Key points

llamay 0.4.15 – signed installers and optional background service

The release notes describe three installer flavours: a macOS .dmg, a Linux one-liner (curl -fsSL https://llamay.com/install.sh | sh) and a Windows .exe or PowerShell one-liner (irm https://llamay.com/install.ps1 | iex). All binaries are now signed – macOS artifacts carry a Developer ID and are notarised, while Windows builds use Azure Trusted Signing. A SHA256SUMS file, itself signed with Sigstore, lets users verify the download integrity. (llamay releases)

The installer also drops a static llamay binary onto the system PATH and, unless you set LLAMAY_NO_SERVICE=1, starts a background server listening on http://127.0.0.1:11435.

The flag is read only by the installer; the running server can be started later with llamay serve and the flag is ignored at that point. This gives users a clear choice: install the binary and let the service run automatically, or keep the binary silent and launch it manually when needed.

Why it matters: For engineers who already run local inference, the signed installers remove a lingering security concern when pulling a binary from the internet.

The optional background service means you can keep the server off during development, avoiding an unwanted open port, and start it only when you are ready to serve the OpenAI, Anthropic or Ollama-compatible APIs. The one-liner installer also aligns with the simplicity of tools like ollama and llama.cpp, making it easy to add llamay to a fresh workstation or CI pipeline.

# Install and start the server automatically
curl -fsSL https://llamay.com/install.sh | sh

# Install but skip the automatic server; start it later when you need it
curl -fsSL https://llamay.com/install.sh | LLAMAY_NO_SERVICE=1 sh
llamay serve -addr 127.0.0.1:11435

Ollama 0.40.13 – LFM3 models now run everywhere

Ollama’s latest patch adds the LFM3 family (including lfm3-audio and lfm3-omni) to its catalogue for Linux, Windows and Apple silicon. The models support text-to-speech and multimodal generation, and they are now available through the same ollama pull workflow that works on all three platforms. (Ollama releases)

The change is on the Ollama side. llamay reads Ollama’s model store, so a model Ollama has already downloaded is served by llamay with no second download and no command to register it — provided its tensor types are ones llamay reads, which is worth checking for a model family this new.

Why it matters: If you are experimenting with multimodal or speech-enabled LLMs, you no longer need a separate GPU-enabled Linux box; the same model runs on a MacBook with Apple silicon or on a Windows laptop. For llamay users, the ability to reuse Ollama’s cache reduces storage duplication and speeds up the first run, as the binary simply points at the existing GGUF file.

# Pull LFM3-omni with Ollama, once
ollama pull lfm3-omni

# llamay finds it in Ollama's store and serves it by name
llamay serve

Ternary Bonsai 2 support in ik_llama.cpp

The ik_llama.cpp fork (a community-maintained variant of llama.cpp) has merged upstream support for the ternary Bonsai 2 quantisation format (PQ2_0). Previously, only the original llama.cpp accepted QK = 256 blocks (TQ2_0), while Prism’s fork used QK = 128, causing incompatibilities when offloading layers to CUDA. The new commits resolve the Hadamard sign-tensor handling, allowing native CUDA offload and fast CPU execution for Bonsai 2 models. (r/LocalLLaMA)

Because llamay reads GGUF files directly and only supports a fixed list of tensor types (including Q4_K, Q8_0, IQ4_NL, etc.), it does not recognise the ternary TQ/PQ formats. Consequently, Bonsai 2 models will be rejected by llamay with an “unsupported type” error. Users who need ternary quantisation must continue to use a llama.cpp-based engine such as ik_llama.cpp or PrismML.

Why it matters: The addition expands the toolbox for engineers seeking lower-precision models that run faster on modest GPUs. If you are already using ik_llama.cpp for Bonsai 2, you can keep benefiting from the new CUDA path. For llamay users, the change does not affect you unless you switch to a supported quantisation format like Q4_K or Q8_0.

llama.cpp b12025 – attention tensor rotation fix

The b12025 patch to llama.cpp corrects a rotation issue in attention tensors for layers that share KV caches. The fix is internal to llama.cpp and does not alter the GGUF file format. Since llamay implements its own inference engine in Go and does not depend on llama.cpp code, this bug-fix has no impact on llamay’s behaviour or performance. (llama.cpp releases)

Why it matters: If you are running models with llama.cpp, you can expect more accurate attention handling after updating to this commit. For llamay, the change is irrelevant; the engine already computes KV caches independently of llama.cpp’s implementation.

QwenPU – an NPU compiled inside Qwen 3.5-4B weights

A Reddit user demonstrated that an NPU implementation can be compiled into the weight tensor of Qwen 3.5-4B, allowing the model to execute another LLM (Karpathy’s stories260k) inside its own forward pass. The technique involves embedding matmul, softmax and attention kernels directly into the weight blob, effectively turning the model into a tiny hardware accelerator for a second model. (r/LocalLLaMA)

The approach is experimental and relies on custom weight manipulation. The experiment is of interest to researchers exploring self-hosting of specialised accelerators, but it does not affect the day-to-day workflow of most local inference engineers.

Why it matters: The demo showcases the flexibility of the GGUF format and hints at future possibilities for on-device hardware emulation. It is a research result rather than something to deploy.

What this means if you run models locally

The most immediate impact for local engineers is the new, signed installer for llamay 0.4.15. It removes a security hurdle and gives you explicit control over whether a background API server starts automatically. If you already use Ollama to fetch large multimodal models such as LFM3, you can now share those GGUF files with llamay without re-downloading, thanks to llamay’s ability to read Ollama’s model store.

Quantisation advances in the llama.cpp ecosystem (ternary Bonsai 2, attention-tensor fixes) broaden the options for users of that engine, but they do not affect llamay because it only supports a defined set of tensor types. If you need those newer formats, keep a llama.cpp-based tool alongside llamay.

Prompt caching in llamay is always on, and /v1/contexts lets you compute a prefix once and fork it explicitly; the new installer does not change either. The KV cache precision flag -kv can still be tuned (f32, q8, vq8) to balance memory usage against accuracy, but it does not enable or disable caching.

A typical workflow after the update might look like this:

# Install llamay with the background service disabled
curl -fsSL https://llamay.com/install.sh | LLAMAY_NO_SERVICE=1 sh

# Pull a GGUF model directly from HuggingFace
llamay pull hf:<repo>/<file.gguf>

# Or pull by name from Ollama's registry
llamay pull qwen2.5:0.5b

# Serve everything in the store (and Ollama's), with a q8 KV cache
llamay serve -kv q8

# Compute a reusable prompt prefix once
curl -X POST http://127.0.0.1:11435/v1/contexts -d '{"messages": [{"role": "system", "content": "You are a helpful assistant."}]}'

In short, the llamay update improves security and installation ergonomics, while the broader ecosystem continues to evolve with new models and quantisation schemes that may or may not be compatible with llamay’s supported tensor set.

FAQ

How do I install llamay without starting the API server automatically?

Run the installer with the environment variable LLAMAY_NO_SERVICE=1, e.g. curl -fsSL https://llamay.com/install.sh | LLAMAY_NO_SERVICE=1 sh. The binary is placed on your PATH and you can start the server later with llamay serve.

Can I use Ollama-downloaded LFM3 models with llamay?

Yes. Ollama stores GGUF files in a known directory, and llamay automatically discovers models already present there, so llamay serve serves it by name without pulling it again, as long as llamay reads its tensor types.

Does the ternary Bonsai 2 support in ik_llama.cpp affect llamay?

No. Ternary Bonsai 2 uses tensor types that are not on llamay’s supported list, so llamay will reject such files. To run Bonsai 2 models you need a llama.cpp-based engine like ik_llama.cpp.

Sources