Hardware for Local AI: What You Need in 2026
“What hardware do I need for local AI?” is the most frequent question — and it has a concrete answer, not a list of cards. The variable that decides almost everything is a single one: the memory where the model weights live (VRAM or unified RAM). Everything else — speed, cost, complexity — is a consequence of that. This guide takes you from budget to choice, with October 2026 prices.
The rule that governs the choice
A model at 4-bit weighs roughly (params ÷ 3.4) GB. Add 1–4 GB of KV cache depending on context, and you have the minimum memory. The VRAM you need:
| Model (Q4_K_M) | Weights | + KV cache 8K | Minimum memory |
|---|---|---|---|
| 8B (Llama 3.1, Qwen3) | ~5 GB | ~6–7 GB | 8 GB |
| 14B (Qwen3) | ~8.3 GB | ~10–11 GB | 12–16 GB |
| 30B MoE (Qwen3-30B-A3B) | ~19 GB | ~21–22 GB | 24 GB |
| 70B (Llama 3.1, Qwen3) | ~40 GB | ~43–45 GB | 48 GB |
Speed depends on memory bandwidth, not on the number of GPUs or cores: an 8B Q4 does 15–40 tok/s on a 400–800 GB/s GPU, 10–20 on a 200 GB/s, 3–10 on DDR5 dual-channel, 2–3 on a Raspberry Pi. If you want the full reasoning, Model without a GPU: when the CPU is truly enough does it with numbers and benchmarks.
The three families of machines
1. NVIDIA GPUs (the standard path)
CUDA is the support standard: Ollama, llama.cpp, vLLM, LM Studio all work without configuration. October 2026 list prices, with the note that Blackwell cards often sell above MSRP:
| Card | VRAM | Bandwidth | Price (Oct 2026) | What it runs |
|---|---|---|---|---|
| RTX 4060 Ti | 16 GB | 288 GB/s | ~$499 (8GB: $399) | 8B–14B Q4–Q5 at useful speed |
| RTX 5060 Ti | 16 GB GDDR7 | 448 GB/s | $429 MSRP, ~$550 street | 8B–14B Q4–Q5, faster |
| RTX 5070 Ti | 16 GB GDDR7 | 896 GB/s | $749 MSRP, ~$1,100–1,180 street | 32B Q4 at practical speed, fast 30B MoE |
| RTX 4090 | 24 GB | 1,008 GB/s | $1,600–2,000 street | 32B Q5–Q8, 70B Q3–Q4 |
| Used RTX 3090 | 24 GB | 936 GB/s | ~$1,000 | The “cheap 24GB” option: 32B Q4 |
The quality tipping point is 24GB: below it you stop at 14B or 30B MoE in Q3; above it, the jump to 32B Q4–Q5 is the sharpest you can make without entering workstation territory. If the budget is one GPU, the 16GB 5070 Ti is the best quality-price balance for the 8–14B tier; the used 24GB 3090 is the pick of anyone who wants 32B without paying triple.
2. Apple Silicon (unified memory)
M chips integrate CPU and GPU on a single memory pool: what matters is how much RAM is in it and how much is allocable to the GPU (on a 32GB Mac, ~24GB; on a 64GB, ~48GB). No configuration, no drivers, and high bandwidth: an M4 Max with 128GB carries a 32B Q4 at practical speed. If you are already in the Apple ecosystem, this is the cleanest path: Local AI on Mac: the complete guide sizes models per chip, and unified memory compared puts Mac numbers against x86 platforms.
3. CPU + RAM (budget or already owned)
If you do not want to spend on a GPU, the CPU carries 8B models at Q4 with 5–15 tok/s on DDR5 dual-channel — enough for light daily use. The costs:
| Option | Indicative price (Oct 2026) | What you get |
|---|---|---|
| 64GB DDR5 mini-PC (e.g. Ryzen 7 8700G + 64GB) | €250–500 | 8B Q4 at 8–15 tok/s, ~€2/month electricity at 24/7 |
| Used: Ryzen 9 5950X + 64GB DDR4 | ~€350–450 | 8B Q4 at 8–12 tok/s, DDR4: lower bandwidth |
| Mac Mini M4 (32GB) | ~$1,100 | 14B Q4 at 15–25 tok/s, 5–15W draw |
The electricity line is the one that surprises: a machine at 10W average at 24/7 does 7.2 kWh/month, i.e. ~€1.80 at €0.25/kWh. The full electricity math is in the costs article.
What it really costs, over 36 months
The question is not “how much does the machine cost”, it is “how much does having model X for 3 years cost”. On an 8B Q4, 10 minutes of chat a day:
| Choice | Purchase | Electricity (36 months) | 36-month total |
|---|---|---|---|
| Mac Mini M4 32GB | ~$1,100 | ~€70 | ~€1,150 |
| PC + RTX 5060 Ti 16GB (complete system) | ~€900 | ~€90 (system at 40W average) | ~€990 |
| PC + used RTX 3090 (added to existing PC) | ~€1,000 | ~€130 (system at 60W average) | ~€1,130 |
| 64GB mini-PC, no GPU | ~€350 | ~€70 | ~€420 |
| Cloud API (GPT-5.6 Terra rate): 10 min/day (~90k tokens/month) | €0 | — | ~€80 (≈ $2.5/month) |
The cloud has no upfront cost, but the bill grows with usage: at 10 minutes a day it stays negligible (~$2.5/month, as shown in What a token costs); at 20 million tokens a month it reaches ~€184/month at the same Terra rate, and that is where the €900–1,000 machine stops costing and the cloud does not. Rule of thumb: light use (under half an hour a day) → cloud; heavy daily use → machine.
The choice by budget
- Zero / you already have a PC with 16–32GB: start with CPU + 8B Q4 model. It works, it is slow, but you learn the workflow. €0.
- ~€500: 16GB GPU (4060 Ti or 5060 Ti). 14B Q4 at 15–30 tok/s: the quality-price balance for most people.
- ~€900–1,000: complete system with 16GB, or used 3090 with 24GB. Here you enter 32B territory: the biggest quality jump.
- ~$1,100 in the Apple ecosystem: 32–64GB Mac. No drivers, no configuration, 5W idle draw.
- Unlimited budget: 24GB+ workstation (5090 or RTX PRO 6000). For people who work on models every day, not for people who use them.
And the limits?
A 16GB GPU does not run a 70B with quality: either drop the quantization (Q3: quality visibly degrades on long text) or change machines. And Blackwell street prices are volatile — MSRP is a promise, street price is the reality, and it must be re-checked at purchase time. The GPU or CPU comparison covers the middle case; how to read model specs is useful when you reach the model's spec sheet.
In short: measure memory first (weights + KV cache), then bandwidth (decides tok/s), and only then price. For most people the answer is a 16GB GPU ($500) or the CPU they already have (€0); the real jump is at 24GB (~€1,000), where 32B becomes daily practice.
Sources
- NVIDIA — GeForce RTX cards (5060 Ti / 5070 / 5070 Ti list prices)
- BestValueGPU — RTX 5070 Ti street-price history (Oct 2026: ~$1,100–1,180)
- BestValueGPU — RTX 4090 price history (Oct 2026: ~$2,000–2,999)
- llama.cpp — benchmark megathread: tok/s per hardware and quantization
- Apple — Mac Mini M4 (configurations and prices)
Article generated with AI
In these reading paths
Enjoyed this article?
Subscribe to the newsletter for new tutorials and guides.