Skip to main content

Blog

Articles, guides, and insights on local artificial intelligence

16 articles found per "vram"

Nemotron 3.5 Bolt: NVIDIA's 74B MoE with ~9B active — what changes on your desktop
ModelsIntermediate

Nemotron 3.5 Bolt: NVIDIA's 74B MoE with ~9B active — what changes on your desktop

74B parameters, 9B active: NVIDIA's MoE for local agents fits 24GB of VRAM via IQ2 quantization. Real VRAM, declared benchmarks and honest limits.

iaLocale Team
November 2, 2026
MOSS-Image: OpenMOSS-Team's ComfyUI ripack, workflows and real requirements
Use casesIntermediate

MOSS-Image: OpenMOSS-Team's ComfyUI ripack, workflows and real requirements

MOSS-Image: OpenMOSS-Team's ComfyUI ripack, ready in two folders. Real-world VRAM computed per weights combination, from ~11GB to ~31GB, plus the four included workflows.

iaLocale Team
October 31, 2026
Local Images: ComfyUI 0.40 and comfy_attention — new for all models
Use casesIntermediate

Local Images: ComfyUI 0.40 and comfy_attention — new for all models

ComfyUI 0.40 extends comfy_attention to all models with per-model selectable kernels and LoRA override, turns on the asset system by default with a local API, and standardizes 3D output; the VRAM table stays the choice reference.

iaLocale Team
October 27, 2026
Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
ModelsIntermediate

Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM

inclusionAI's Ling 3.1 Flash: hybrid MoE 560B total / 25B active, native 1M-token context, FP8 checkpoint, MIT license. What it concretely means for 24GB of VRAM: quantizations, parallelism and API costs.

iaLocale Team
October 19, 2026
Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally
ModelsIntermediate

Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally

Aleph Alpha's Kolibri (78.1B/3.46B, Apache 2.0, FP8) is the European open-weight MoE: reasoning, tool calling and context up to 1M tokens. In FP8 it weighs ~78GB of VRAM: it runs on an RTX 5090 32GB + shared RAM or on unified-memory Macs; for practical adoption the kolibri1 architecture in llama.cpp and Ollama/LM Studio builds are still missing.

iaLocale Team
October 18, 2026
NVIDIA, AMD and Intel for local AI: which platform to choose
ComparisonsIntermediate

NVIDIA, AMD and Intel for local AI: which platform to choose

CUDA, ROCm or Vulkan? We compare the three local-AI platforms based on what is actually available: official software stacks, cards by VRAM tier (12, 16 and 24GB) and real speeds. Where the obvious choice remains, where AMD wins on value, and when it makes sense to leave NVIDIA.

iaLocale Team
October 14, 2026
The 16GB Blackwell cards: 5060 Ti, 5070, 5070 Ti
HardwareIntermediate

The 16GB Blackwell cards: 5060 Ti, 5070, 5070 Ti

The 16GB VRAM tier is the center of gravity of local AI in 2026. It's enough memory to run the best open dense models in Q4 formats — the 27B class, the new balance point of the open catalog — without giving up too much context, and a list price that (depending on the month) stays within a personal budget. The Blackwell generation offers three cards exactly in the middle of this tier: the RTX 5060 Ti, 5070 and 5070 Ti. Same generation, same GDDR7, three very different experiences. Here's how they compare: what each one really brings you, and where your budget should go.

iaLocale Team
October 13, 2026
Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB
ComparisonsIntermediate

Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB

A 27.8B dense or a MoE with 3.8B active, a 262K context against 256K and almost the same weight at Q4_K_M: two opposite ways of putting big-model quality on a 24GB consumer card. Which is faster, which goes deeper, and when to pick one over the other.

iaLocale Team
October 9, 2026
Local Image Models: from SDXL to Qwen-Image
ModelsIntermediate

Local Image Models: from SDXL to Qwen-Image

From SDXL's 2.6B parameters to Qwen-Image 2.1's 7B: how diffusion models work, how much VRAM you actually need, and which model to pick for your hardware.

iaLocale Team
October 9, 2026
GGUF: The Format That Keeps Models on Your Hardware
ModelsIntermediate

GGUF: The Format That Keeps Models on Your Hardware

GGUF is the single-file format that ships models for local inference: a header with a signature and version, key-value metadata, a tensor index, and weights aligned for fast mmap loading. Quantization types with real sizes (8B from 14.96 GiB at F16 to 4.58 GiB at Q4_K_M), the filename convention, and the rule for picking a size against your VRAM or RAM.

iaLocale Team
September 30, 2026
Fine-tuning: adapting a model to your data with LoRA and QLoRA
GuidesIntermediate

Fine-tuning: adapting a model to your data with LoRA and QLoRA

Fine-tuning with LoRA and QLoRA: when it's truly needed, how to prepare 100-500 quality examples, how much VRAM it takes (8-12 GB in QLoRA for a 7B), and which local tools to use: LLaMA-Factory, Unsloth, MLX.

iaLocale Team
September 29, 2026
Qwen-Image 2.1: Generate and Edit Images Locally
ModelsIntermediate

Qwen-Image 2.1: Generate and Edit Images Locally

Qwen-Image 2.1 unifies text-to-image, editing, and RGBA transparency in a 7B open model (32-layer DiT). Local setup with diffusers, native ComfyUI support, VRAM needs (~14GB in bfloat16), and use cases (stickers, product photos, design workflows).

iaLocale Team
September 28, 2026
Quantization: Get More From the Same Hardware
ModelsIntermediate

Quantization: Get More From the Same Hardware

Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).

iaLocale Team
September 28, 2026
When a Big Model Isn't Needed: Choosing by Size
ModelsIntermediate

When a Big Model Isn't Needed: Choosing by Size

The method for choosing model size by hardware: the three limits (memory, speed, perceived latency), the VRAM table for Q4 quantization, the hardware classes, and the cases where the small model wins.

iaLocale Team
September 27, 2026
Mistral vs Gemma: Light Models Compared
ComparisonsBeginner

Mistral vs Gemma: Light Models Compared

Full comparison of the Mistral and Gemma open model families: sizes, required VRAM, context, supported languages and licenses, with a practical method for picking the right model for your hardware.

iaLocale Team
September 25, 2026
GPU or CPU: when the CPU is enough
HardwareIntermediate

GPU or CPU: when the CPU is enough

How prefill and decode work, what matters for speed (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure.

iaLocale Team
September 21, 2026