Skip to main content

Blog

Articles, guides, and insights on local artificial intelligence

23 articles found per "hardware"

Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active
ModelsAdvanced

Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active

The first Qwen-Max-class open-weight model (2.4T parameters, 95B active) outgrows any consumer node: the math on weights, multi-node hardware, 36-month amortization, and power costs, and when running it locally makes sense.

iaLocale Team
November 22, 2026
Cosmos 3 Edge: the DROID policy for robotics that runs on the edge
ComparisonsIntermediate

Cosmos 3 Edge: the DROID policy for robotics that runs on the edge

Cosmos3-Edge-Policy-DROID: NVIDIA's 4B robotic policy. How much hardware you need to run it in real time on a Jetson, and where it stops.

iaLocale Team
October 30, 2026
Where local models learn: open weights explained without the fluff
FundamentalsBeginner

Where local models learn: open weights explained without the fluff

The model doesn't hook into the internet for its abilities: it absorbed them once at pre-training and executes them on your hardware. Open weights isn't open data, and the models come from many labs, not one.

iaLocale Team
October 27, 2026
Nemotron-Parse: NVIDIA's OCR family for reading documents locally
ModelsBeginner

Nemotron-Parse: NVIDIA's OCR family for reading documents locally

Nemotron Parse is NVIDIA's vision encoder-decoder family that reads PDFs locally: formatted markdown, LaTeX formulas, coordinates and a semantic class for every element, in reading order. v1.1 (Nov 2025), v1.2 (Feb 2026) and 2.0 (Jun 2026), all under 1 billion parameters and re-released on October 20; 2.0 reaches 0.72 ANLS on Indic scripts. Locally with vLLM v0.20 through v0.26: your documents stay on your hardware.

iaLocale Team
October 24, 2026
Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup
GuidesAdvanced

Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup

What actually fits in the 512GB Mac Studio M5 Ultra: the 4-bit rule for MoE (active vs total params), three GGUF setup paths (Ollama, LM Studio, llama.cpp), why the KV cache is the real bottleneck, and when the $9,999 pays back against APIs.

iaLocale Team
October 20, 2026
Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
ModelsIntermediate

Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM

inclusionAI's Ling 3.1 Flash: hybrid MoE 560B total / 25B active, native 1M-token context, FP8 checkpoint, MIT license. What it concretely means for 24GB of VRAM: quantizations, parallelism and API costs.

iaLocale Team
October 19, 2026
Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally
ModelsIntermediate

Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally

Aleph Alpha's Kolibri (78.1B/3.46B, Apache 2.0, FP8) is the European open-weight MoE: reasoning, tool calling and context up to 1M tokens. In FP8 it weighs ~78GB of VRAM: it runs on an RTX 5090 32GB + shared RAM or on unified-memory Macs; for practical adoption the kolibri1 architecture in llama.cpp and Ollama/LM Studio builds are still missing.

iaLocale Team
October 18, 2026
Mac Studio M5 Ultra: 512GB of unified memory on the desktop
HardwareIntermediate

Mac Studio M5 Ultra: 512GB of unified memory on the desktop

Four dies, 512GB of unified memory at 1.2 TB/s: the Mac Studio M5 Ultra runs 100-560B MoE models locally. The speed math, the cost comparison with the DGX Station, and when the Mac is the right choice.

iaLocale Team
October 16, 2026
What a token costs: five routes, the math, and the break-even point
ComparisonsIntermediate

What a token costs: five routes, the math, and the break-even point

Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.

iaLocale Team
October 15, 2026
Unified memory: from Strix Halo to Mac Studio M5 Ultra — what fits and at what speed
HardwareIntermediate

Unified memory: from Strix Halo to Mac Studio M5 Ultra — what fits and at what speed

128GB or 512GB of memory shared between CPU and GPU: three real platforms, the bandwidth that determines speed, and the math on what fits and for whom it makes sense.

iaLocale Team
October 10, 2026
Local Image Models: from SDXL to Qwen-Image
ModelsIntermediate

Local Image Models: from SDXL to Qwen-Image

From SDXL's 2.6B parameters to Qwen-Image 2.1's 7B: how diffusion models work, how much VRAM you actually need, and which model to pick for your hardware.

iaLocale Team
October 9, 2026
Local voice: from Whisper to text-to-speech
GuidesIntermediate

Local voice: from Whisper to text-to-speech

How to build a fully local voice pipeline: transcription with Whisper, synthesis with Piper or Kokoro-82M, hardware requirements, and practical use cases.

iaLocale Team
October 5, 2026
GGUF: The Format That Keeps Models on Your Hardware
ModelsIntermediate

GGUF: The Format That Keeps Models on Your Hardware

GGUF is the single-file format that ships models for local inference: a header with a signature and version, key-value metadata, a tensor index, and weights aligned for fast mmap loading. Quantization types with real sizes (8B from 14.96 GiB at F16 to 4.58 GiB at Q4_K_M), the filename convention, and the rule for picking a size against your VRAM or RAM.

iaLocale Team
September 30, 2026
Quantization: Get More From the Same Hardware
ModelsIntermediate

Quantization: Get More From the Same Hardware

Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).

iaLocale Team
September 28, 2026
When a Big Model Isn't Needed: Choosing by Size
ModelsIntermediate

When a Big Model Isn't Needed: Choosing by Size

The method for choosing model size by hardware: the three limits (memory, speed, perceived latency), the VRAM table for Q4 quantization, the hardware classes, and the cases where the small model wins.

iaLocale Team
September 27, 2026
Mistral vs Gemma: Light Models Compared
ComparisonsBeginner

Mistral vs Gemma: Light Models Compared

Full comparison of the Mistral and Gemma open model families: sizes, required VRAM, context, supported languages and licenses, with a practical method for picking the right model for your hardware.

iaLocale Team
September 25, 2026
Local AI for your business: 5 practical use cases
Use casesBeginner

Local AI for your business: 5 practical use cases

Five practical use cases of local AI for businesses: customer support, documents, internal search, CRM and generation — with the tools, minimum hardware and a 30-minute starting plan.

iaLocale Team
September 22, 2026
GPU or CPU: when the CPU is enough
HardwareIntermediate

GPU or CPU: when the CPU is enough

How prefill and decode work, what matters for speed (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure.

iaLocale Team
September 21, 2026
What is local AI and why you need it
FundamentalsBeginner

What is local AI and why you need it

A practical explanation of local AI: running a model on your own hardware instead of the cloud. Differences from cloud AI, the concrete benefits (availability, privacy, predictable cost, full control), honest limits, and a five-minute start with Ollama.

iaLocale Team
September 8, 2026
Llama 3 and Llama 4: Meta's open family — which model runs on your hardware
ModelsBeginner

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware

Llama 3 and 4 in a table: 3B/8B/70B and Scout MoE 109B, RAM required per model, the Community license and the choice by budget.

iaLocale Team
June 10, 2026
Hardware for Local AI: What You Need in 2026
HardwareIntermediate

Hardware for Local AI: What You Need in 2026

The memory rule (weights + KV cache), real October 2026 prices for GPUs, Apple Silicon and CPU, the 36-month cost math and the choice by budget.

iaLocale Team
June 10, 2026
Running LLM Models Locally: From First Install to First Working Model
GuidesIntermediate

Running LLM Models Locally: From First Install to First Working Model

Learn how to install and use large language models on your own computer, without relying on cloud services.

iaLocale Team
June 10, 2026
Local AI on Raspberry Pi
HardwareIntermediate

Local AI on Raspberry Pi

From first install to a model that answers on a Pi 5: real 2026 prices, the tok/s = bandwidth ÷ file formula, the electricity bill and the tricks that matter.

iaLocale Team
June 10, 2026