Blog
Articles, guides, and insights on local artificial intelligence
14 articles found per "gpu"

NVIDIA's Cosmos3-H-Dreams: real-time surgical simulation, and local robotics
A 15B world model that generates robotic surgery video in real time on a single GPU: what it changes for local robotics.

ML Drift: Google's open-source on-device GPU engine, and when it beats the generalists
The engine that runs models inside Chrome and YouTube is now open source: architecture, benchmark numbers, and an honest comparison with Ollama and llama.cpp.

MagpieTTS 357M: 12-language speech synthesis that runs on a laptop
NVIDIA releases MagpieTTS Multilingual, 364M parameters and 12 languages in one model: local text-to-speech with a 332 MB GGUF file, no GPU. The C++ runtime NeMo-Speech.cpp takes you from zero to first audio in three commands; the five voices are the same in every language, and for a natively Italian voice Piper is still ahead.

Decision models in local: JEV-35B and the 3B-active System 1
You don't need a big LLM to decide. JEV-35B (35B, 3B active) in NVFP4 on a 32GB GPU and ej (11.4MB) on CPU show how to do routing, classification and quality control without generating a single token.

What a token costs: five routes, the math, and the break-even point
Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.

NVIDIA, AMD and Intel for local AI: which platform to choose
CUDA, ROCm or Vulkan? We compare the three local-AI platforms based on what is actually available: official software stacks, cards by VRAM tier (12, 16 and 24GB) and real speeds. Where the obvious choice remains, where AMD wins on value, and when it makes sense to leave NVIDIA.

The 16GB Blackwell cards: 5060 Ti, 5070, 5070 Ti
The 16GB VRAM tier is the center of gravity of local AI in 2026. It's enough memory to run the best open dense models in Q4 formats — the 27B class, the new balance point of the open catalog — without giving up too much context, and a list price that (depending on the month) stays within a personal budget. The Blackwell generation offers three cards exactly in the middle of this tier: the RTX 5060 Ti, 5070 and 5070 Ti. Same generation, same GDDR7, three very different experiences. Here's how they compare: what each one really brings you, and where your budget should go.

Unified memory: from Strix Halo to Mac Studio M5 Ultra — what fits and at what speed
128GB or 512GB of memory shared between CPU and GPU: three real platforms, the bandwidth that determines speed, and the math on what fits and for whom it makes sense.

Costs: local vs API, done with real numbers
How much does local really cost compared to a cloud API? Verified per-token prices, GPU amortization, electricity and the break-even point, calculated with real numbers.

A Model Without a GPU: When the CPU Is Actually Enough
In 2026, CPU-only inference is no longer a fallback: when it genuinely works, what speed to expect, and which levers to pull to squeeze the most out of your processor.

GPU or CPU: when the CPU is enough
How prefill and decode work, what matters for speed (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure.

vLLM: serving models fast
vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the key concepts for choosing when to use it instead of Ollama.

Local AI on Mac: the complete Apple Silicon guide
Why Apple Silicon (unified memory, GPU cores) is the simplest machine for local AI: the RAM you need, installing Ollama, recommended models per chip generation, and realistic performance expectations.

Hardware for Local AI: What You Need in 2026
The memory rule (weights + KV cache), real October 2026 prices for GPUs, Apple Silicon and CPU, the 36-month cost math and the choice by budget.

