What is this?
ggml is a plain C/C++ tensor library built for running machine learning models with minimal setup, no external dependencies, and support for CPU, GPU, NPU, and browser backends. It's the core engine that powers projects like llama.cpp.
ggml is a plain C/C++ tensor library built for running machine learning models with minimal setup, no external dependencies, and support for CPU, GPU, NPU, and browser backends. It's the core engine that powers projects like llama.cpp.Is this a standalone app or a library I build on top of?It's a low-level C/C++ tensor library, not an application — you link it into your own project or use it through tools like llama.cpp that embed it.
It's a low-level C/C++ tensor library, not an application — you link it into your own project or use it through tools like llama.cpp that embed it.How do I build it?You build ggml from source; the repo includes a minimal, fully commented matrix-multiplication example in examples/simple to get started.
You build ggml from source; the repo includes a minimal, fully commented matrix-multiplication example in examples/simple to get started.Do I need any external libraries to use it?No — ggml is implemented in plain C/C++ with zero external dependencies.
No — ggml is implemented in plain C/C++ with zero external dependencies.Can I submit changes directly to this repo?Yes, but changes to the core ggml library (including the CMake build system) are expected to go through the llama.cpp repo instead, since that gets more visibility, testing, and review.
Yes, but changes to the core ggml library (including the CMake build system) are expected to go through the llama.cpp repo instead, since that gets more visibility, testing, and review.What hardware and platforms does it run on?ggml runs across x86, ARM, RISC-V, LoongArch, PowerPC, and s390x CPUs plus WebAssembly, with backend support for CPU, GPU, NPU, and even inside the browser.
ggml runs across x86, ARM, RISC-V, LoongArch, PowerPC, and s390x CPUs plus WebAssembly, with backend support for CPU, GPU, NPU, and even inside the browser.What quantization formats does ggml support?It supports 2- to 8-bit integer quantization plus the newer MXFP4 and NVFP4 microscaling formats, letting models run in much less memory.
It supports 2- to 8-bit integer quantization plus the newer MXFP4 and NVFP4 microscaling formats, letting models run in much less memory.Does ggml allocate memory while running?No — it's designed for zero memory allocations during runtime, which keeps inference predictable and fast.
No — it's designed for zero memory allocations during runtime, which keeps inference predictable and fast.What is GGUF and where can I learn the internals?GGUF is ggml's model file format; the docs also cover an 'Introduction to ggml' overview and a 'GGML tips & tricks' guide for working with the library.
GGUF is ggml's model file format; the docs also cover an 'Introduction to ggml' overview and a 'GGML tips & tricks' guide for working with the library.Why is ggml often mentioned alongside llama.cpp?llama.cpp is built on top of ggml as its tensor backend, and ggml's own maintainers route core-library contributions back into the llama.cpp repo for review.
llama.cpp is built on top of ggml as its tensor backend, and ggml's own maintainers route core-library contributions back into the llama.cpp repo for review.What are similar projects?cpu-performance-engineering, llama.cpp, BitNet, simdjson, DeepGEMM, tilelang, FlashMLA, DeepGEMM-Ascend
cpu-performance-engineering, llama.cpp, BitNet, simdjson, DeepGEMM, tilelang, FlashMLA, DeepGEMM-AscendWhich topics is it in?tensor, risc-v, simd, cpu, x86, arm, webassembly, low-level, llm-inference, gpu-acceleration, quantization, wasm
tensor, risc-v, simd, cpu, x86, arm, webassembly, low-level, llm-inference, gpu-acceleration, quantization, wasm