GGUF

GGML Unified Format. The standard file format for distributing quantized LLM models, especially with llama.cpp. It supports storing the model, quantized weights, and tokens. It is the predominant format for local inference on consumer hardware.