Yzma: embeddings up to 2.6x faster than Ollama, the comparison

Yzma is an inference engine written in Go that does one thing that is easy to say and rare to pull off: it calls llama.cpp inside its own process. No separate server, no REST, no waiting. The result, per the numbers the team published, is an engine that produces embeddings up to 2.6x faster than Ollama. In this article we take the reference benchmarks, redo the math ourselves, and see what changes when the embedding becomes the bottleneck of your RAG.

What Yzma is and why it should matter to you

Yzma is an open-source inference engine written in pure Go. It does not reimplement the engine: it embeds llama.cpp in-process via purego/ffi, without CGo. It reads GGUF models and runs on CUDA, Metal, Vulkan and OpenVINO, all the way to the browser with WebGPU. It covers roughly 96% of llama.cpp's functionality, with peaks of 840 tok/s on a desktop workstation and 30 tok/s on an Arduino board.

For those who have never heard of an inference engine: it is the software that runs the model. Ollama is one of these engines — a server that exposes a local API. Yzma positions itself differently: it is a library you put inside your Go application, with the option of an OpenAI-compatible server if you want one.

The architectural difference is the whole point: when your code calls an embedding, with Ollama the request travels to a server, crosses the network, the server decodes the request, llama.cpp computes, the vector comes back and the server wraps it again. With Yzma, the vector is born inside your process and lands directly in your variable.

The numbers: embeddings 2.6x faster

The reference benchmarks are published on the project's site, collected on 16 October 2026 on a laptop with an Intel Core i9-13900HX and an RTX 4070. Versions used: yzma 1.30.0, Ollama 0.40.5, Docker Model Runner v1.2.8. Embedding model: bge-small, 29 prompt tokens, 384-dimension vector.

Engine	Latency per embedding	Prompt throughput	Indexing 10,000 documents
Yzma	1.7 ms	17,123 tok/s	17 s
Ollama	4.5 ms	6,393 tok/s	45 s
Docker Model Runner	4.6 ms	6,360 tok/s	46 s

We redo the math ourselves, without trusting the table:

• Latency ratio: 4.5 ms ÷ 1.7 ms = 2.65. The claimed "2.6x" checks out exactly.
• Throughput ratio: 17,123 ÷ 6,393 = 2.68. Same result from the other side.
• For an index of 10,000 documents: 10,000 × 1.7 ms = 17 s with Yzma; 10,000 × 4.5 ms = 45 s with Ollama. An index that takes 45 seconds takes a quarter of the time with the more popular engine.

The same page also measures text generation, where the advantage narrows: 9.2–12.1% more in speed and a time-to-first-token between 48% and 60% of the server's. With Gemma 4: 118.6 tok/s against 105.8 (ttft 12.3 ms against 24.6). With Qwen3-VL: 166.9 tok/s against 152.8 (ttft 9.0 against 15.1).

Why in-process wins on embeddings

The reason is entirely mechanical. In a server, the REST round trip exists regardless of what the model returns. For text generation the round trip is nearly free: the token arrives via streaming and the network transfer is a tiny fraction of the total time. For embeddings, no: the answer is a single vector, maybe 384 numbers. The model works for under 2 milliseconds, and the fixed part — HTTP, serialization, request decoding — weighs the same amount.

In numbers: if the REST round trip costs ~2.8 ms and the computation ~1.7 ms, the server totals 4.5 ms and in-process 1.7 ms. 62% of Ollama's time in that measurement is not the model: it is the plumbing.

The practical consequence concerns anyone building RAG: indexing is the point where embeddings are called thousands of times, one after another. A quarter of the indexing time is setup time refunded, and a 1.7 ms latency makes it sensible to do more embeddings at query time. The broader comparison between the two engines — architecture, formats, when to choose which — we already did in Ollama vs llama.cpp: under the hood; and if you want to understand where your embeddings end up once computed, start from RAG: ask questions to your own documents.

How to use it

Yzma is first of all a library for Go services. Three commands:

go install github.com/hybridgroup/yzma@latest
yzma install --lib
export YZMA_LIB=$HOME/yzma

Then, in your Go code, the library loads the GGUF model and returns the vector of your call, without starting any other process. If you prefer a server: yzma starts one with an OpenAI-compatible API, so an existing client — Python, Node, curl — points at localhost instead of Ollama.

If it does not work: if the go install line cannot find the toolchain, update Go; if the server does not answer on the local port, check that the YZMA_LIB variable really points at the installed library — it is the most common failure reason at setup.

And the limits?

• The numbers are published by the yzma team, on a single reference machine. No independent lab has replicated the measurement: treat them as declared, not verified.
• Yzma is not a drop-in replacement for Ollama: you do not inherit the model library, the ecosystem and the weekly updates. It is an alternative for those building services in Go who want the shortest path to embeddings.
• The advantage on generated text is marginal (9–12%). If your case is a chatbot, the Yzma argument stands only on time-to-first-token, not on throughput.

Sources

• Official project benchmarks
• Yzma GitHub repository
• Hacker News thread of 19 October 2026

In summary

Yzma embeds llama.cpp in the process, with no REST in between: on the benchmark published by the team (16 October 2026, i9-13900HX + RTX 4070), embeddings drop from 4.5 ms to 1.7 ms, which is 2.6x — and the math holds: 4.5 ÷ 1.7 = 2.65. The 10,000-document index goes from 45 to 17 seconds. On generated text the advantage is 9–12% with a time-to-first-token halved. It is open source, in Go, and covers roughly 96% of llama.cpp: the numbers were published by the vendor on a single machine, so before deciding, measure on your hardware.

Article generated with AI