Skip to main content

Local AI Glossary

Terms and definitions of local artificial intelligence, open source models, and computing.

18 terms in Architectures

Gated DeltaNetArchitectures

A linear-attention variant used in Qwen 3.5/3.8 models: it keeps a constant-size state updated by a delta-style rule (it removes stale information before adding the new one) instead of accumulating all key-value pairs. With few layers of this kind, KV cache memory grows little as the context grows, and classic attention layers alternate with them — for example 1 in 4 in Qwen3.8-2.4T-A95B.

ggmlArchitectures

The C/C++ tensor library that llama.cpp is built on (named after GGML, the format that made it famous). It defines tensors, operations and backends — CPU, GPU, Apple Silicon — on which models run in local inference. It is the hardware layer of the inference stack: runtimes like Ollama and llama.cpp sit on top of it.

GQA (Grouped-Query Attention)Architectures

Attention scheme where several query heads share the same group of Key and Value matrices, reducing the KV-head count compared to MHA. Lowers the cost and memory-access speed during decoding with a minimal quality trade-off.

Linear attentionArchitectures

Family of attention schemes (DeltaNet, Gated DeltaNet, Mamba, etc.) that replace the linearly growing KV cache with a fixed-size recurrent state, drastically reducing memory at long context. It's the technique behind hybrid-attention models like Qwen3.8-27B.

LiteRTArchitectures

Google's on-device AI toolkit, the evolution of TensorFlow Lite. ML Drift integrates with LiteRT as the GPU accelerator: it is the entry path of the engine into mobile apps, the browser, and edge devices.

MetalArchitectures

Apple's GPU compute and graphics API. It is the backend through which on-device AI runs on iPhone, iPad, and Mac: cross-platform engines like ML Drift use it across the whole Apple platform.

Mixture-of-Experts (MoE)Architectures

Model architecture in which the model counts many billions of parameters in total, but for each token activates only a small subset (the 'experts' selected by a router). The result is the quality of a large model with the speed and compute cost of a small one. Example: Qwen3 30B-A3B has 30.5B total parameters but uses only 3.3B per token. The price is that the whole model weight must still be loaded into memory.

N-gram embeddingArchitectures

A mechanism in which the model indexes short runs of consecutive tokens (bigrams, trigrams) through an embedding table, instead of treating them one by one. It cuts computational cost and memory for very long contexts, and is the scaling axis used by Qwen3.8-Flash-Next (51B of dedicated parameters).

OpenCLArchitectures

Open framework for parallel computing on GPUs, CPUs, and other processing units. It is the traditional backend for on-device inference on Android: ML Drift uses it on the Google platform alongside Metal (Apple) and WebGPU (browser).

RAGArchitectures

Retrieval Augmented Generation. Architecture that combines a language model with a document retrieval system. Instead of relying solely on model knowledge, the system retrieves relevant documents from a database before generating the response. This reduces hallucinations and makes the system updatable without retraining.

RetrievalArchitectures

The stage of a RAG system where, from the user's question, the system searches the vector database for the most similar text blocks and adds them to the prompt. The quality of retrieval is the factor that determines the quality of the final answer.

Thinking modeArchitectures

Feature introduced in Qwen3 where the model can generate a long intermediate reasoning step before producing the final answer. It improves quality on logic, math and planning problems, at the cost of more tokens (and therefore more time and memory) per answer. It can be switched on or off, and in Qwen3 the model also decides for itself whether to reason.

TransformerArchitectures

Neural network architecture introduced in 2017 (Attention Is All You Need) that revolutionized NLP. Unlike recurrent networks, Transformers use attention to process the entire input simultaneously. All modern LLMs (GPT, Llama, Mistral, Claude) are based on this architecture.

UltraFusionArchitectures

Apple's technology that fuses two M5 Max dies into a single M5 Ultra chip, presented to the system as one processor. The unified memory of both dies forms a single pool (up to 512GB) accessible to CPU, GPU and Neural Accelerators.

Unified memoryArchitectures

The Apple Silicon architecture in which CPU and GPU share the same pool of RAM. For local AI, the entire model can reside in that memory without copying data between system and video memory.

Vision-Language-Action (VLA)Architectures

A model architecture that takes visual and language input and produces actions directly — keyboard and mouse movements, robot commands — in real time. Trained end-to-end on trajectories (miHoYo's Gamer uses 10K+ hours of gameplay), it is the class behind GUI agents and robotics policies that act by watching the screen, with no APIs or privileged state.

VITSArchitectures

Voiced Language Model with a Varying-rate adversarial (VITS): a neural text-to-speech architecture that generates the waveform directly from text without a separate vocoder. It underpins Piper and Kokoro-82M.

WebGPUArchitectures

Browser API for GPU rendering and compute, the W3C standard replacing WebGL for heavy compute workloads. In 2026 it is the backend through which on-device inference engines (like ML Drift, via Dawn) run directly in the web page, without the cloud.