The llama.cpp project announced in GitHub release notes on 10 November that its software can now feed streamed audio to Gemma 4. llama.cpp is a program for running language models on your own computer. The change is in pull request #31965 and is included in build b12237.

With streaming, the model receives audio as a continuous stream rather than a finished file. The change affects the mtmd library, which handles input that accompanies text in llama.cpp, such as audio. The audio stream advances in 480-millisecond steps, the release notes said. They do not mention latency or explain how to enable the audio stream in the server.

The same change includes a fix to the llama.cpp server. The server’s realtime slot forced the model to return embeddings, the model’s internal state, for every row of an input batch. But only a model that generates audio reserves memory for these outputs. As a result, Gemma 4, a chat model using an audio stream, crashed on its very first batch of audio.

The server now turns embeddings on only for models that generate audio. Only the audio generation pipeline reads the embeddings, the release notes said. The notes indicate that Gemma 4 is not an audio-generating model, so the change concerns audio input.

The release comes with prebuilt packages for macOS, iOS, Linux, Android and Windows, and for compute backends such as CUDA, Vulkan, ROCm, SYCL and OpenVINO. The macOS KleidiAI build and the openEuler packages are missing from this release. Users can download the packages from the project’s GitHub page.