Skip to main content

Local AI Glossary

Terms and definitions of local artificial intelligence, open source models, and computing.

11 terms in Techniques

ChunkingTechniques

The splitting of a document into small text blocks before indexing. Each block becomes a searchable entry in the vector database: block size is one of the first parameters to tune when RAG results are not convincing.

Context compressionTechniques

Technique to shrink the part of the context sent to the model at a reasoning step, dropping or summarizing retrieved or historical portions the current step does not use. Examples: summaries generated by another model (ACON, Context-Folding), token removal via a learned policy (LLMLingua-2), or chunk selection based on the model's own attention (QRAE). On multi-step agents it cuts tokens and latency without, when done well, degrading quality.

EAGLETechniques

Family of speculative decoding techniques (EAGLE paper, arXiv:2401.15077): a small draft model reads the big model's hidden states and predicts the next tokens from there. Sampling stays lossless, like classical speculative sampling. Official EAGLE checkpoints (for example Mistral-Medium-3.5-128B-EAGLE) are served natively by vLLM and SGLang.

Model distillationTechniques

A technique to transfer the capabilities of a large model (the teacher) to a smaller one (the student), which imitates the teacher's behavior instead of repeating its training. The result is a smaller, faster model with a small quality drop: in an extreme case, distilling a 35-step bidirectional video generator into a 4-step causal student is what lets Cosmos3-H-Dreams run in real time.

Prompt EngineeringTechniques

The art of writing effective instructions to get better results from a language model. Includes techniques like few-shot prompting (providing examples in the prompt), chain-of-thought (asking the model to reason step by step) and role prompting (defining a role for the model).

Quantizzazione-Aware Training (QAT)Techniques

Training that simulates quantization during training, so the model learns to work with low-precision weights. Unlike post-training quantization (applied cold, e.g. Q4_K_M), QAT preserves much more quality at the same bit width. Google released official QAT checkpoints for the Gemma 4 family in June 2026.

Scaling lawsTechniques

Mathematical relationship between model or data scale and resulting quality: parameters, tokens and compute grow predictably relative to loss. In the synthetic-data context, scaling laws measure how many tasks it takes to reach a given success rate.

Semantic similarityTechniques

The closeness between two texts' vectors based on meaning, not on literal match. It is the principle behind search in a vector database: "brake" and "braking system" are not the same word, but their vectors are neighbors.

Synthetic dataTechniques

Data generated by an artificial process instead of collected from the real world: synthetic tasks, synthetic images, simulation traces. Used for training when real data costs money, changes, or can't be touched; quality depends on how well the synthetic distribution imitates the real one.

System promptTechniques

The set of instructions that defines a model's permanent behavior: tone, role, response format, constraints. Unlike the user prompt (a single question), the system prompt governs every response of the session and is not shown to the model as a request.

Voice Activity DetectionTechniques

Voice Activity Detection: a technique that identifies which portions of an audio stream contain speech, so the audio can be segmented into relevant chunks and silence ignored. Essential in real-time transcription pipelines.