Capabilities

Speech to text

llamay turns speech into text on this machine with Whisper models, read in pure Go: from the command line, from an OpenAI-shaped HTTP route, and from the microphone in Studio. The audio never leaves the machine.

file.wav any rate, any channels 16 kHz mono resampled 30 s windows log-mel spectrogram Whisper encoder + decoder text greedy, temperature 0 The windows are the only time structure in the output: there are no word or phrase timestamps.
The whole path, from file to text, inside one process.

Get a model

llamay reads whisper.cpp's model files, ggml-*.bin: from ggml-tiny.en.bin (75 MB) to ggml-large-v3-turbo.bin, and their q5_0 and q8_0 quantisations. They are not GGUF, so llamay pull does not fetch them; download one directly: 

curl -LO https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin

.en models are English only and more accurate at English. The others detect the language and can translate into English.

From the command line

llamay transcribe -m ggml-tiny.en.bin speech.wav
And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.

An 11-second clip, in 0.8 seconds on an Apple M4 CPU with the tiny English model.

FlagDoes
-mThe Whisper model file.
-langThe spoken language's code (en, de, fr…). Empty detects it, on a multilingual model.
-translateWrite English whatever language is spoken. Multilingual models only.
-jsonPrint the result as JSON, with each 30-second window's text.
-tWorker threads. 0 picks the performance cores.
-vReport the model, the audio and the time taken.

Over HTTP

Start the server with -whisper and it answers OpenAI's transcription routes. The OpenAI SDK's client.audio.transcriptions.create works unchanged. 

llamay serve -whisper ggml-base.en.bin
curl -s localhost:11435/v1/audio/transcriptions -F [email protected] -F model=whisper-1
{"text":"And so my fellow Americans ask not what your country can do for you, ask what you can do for your country."}
Form fieldAccepted
fileRequired. The WAV audio, up to 32 MiB.
modelEmpty, the served model's name, or any name starting with whisper (so whisper-1 works).
languageA language code, or empty to detect.
response_formatjson (default), text, verbose_json, srt or vtt.
temperatureOnly 0: decoding is greedy.
promptAccepted and ignored.
timestamp_granularities[]Refused with a 400.

POST /v1/audio/translations takes the same form and writes English. The response carries X-Llamay-Transcribe-Ms, the time the transcription took.

Dictation in Studio

With -whisper, Studio's microphone button sends what you say to this server's /v1/audio/transcriptions. Without it, Studio falls back to the browser's own speech recognition, which in Chrome and Edge sends the audio to Google or Microsoft. The button says which one is in use. See the apps.

Limits, stated plainly