News
Latest news and updates on local artificial intelligence
179 news items
KIWI: sub-microsecond GPU-to-CPU invocation, up to 2.3x faster expert-parallel communication
November 22, 2026The KIWI paper (arXiv) flips the device-initiated communication hierarchy: reverse offload on host threads, exposed as a programmable interface, beats GPUDirect backends — up to 2.3x lower expert-parallel MoE communication latency on AMD GPUs and -29% time per token in vLLM.
WAM-Policy: world action models and on-policy distillation for quadrupedal loco-manipulation
November 22, 2026The WAM-Policy paper teaches arm-less quadrupeds to walk on three legs while manipulating with the fourth, following language instructions: on-policy distillation from world action models lets the policy follow commands, and the project is open-sourced.
spiruxbench: 22 local LLMs tested to find the Qwen 3.6 35B successor
November 22, 2026An independent benchmark measures 22 local LLMs (the specific quantizations that run on a gaming PC) with agentic coding challenges in isolated containers — ~10 hours per full run — and compares Qwen 3.6 fine-tunes against alternative dense models like Qwen 3.8 Flash Next.
Riva-Translate 4B v1.1: local translation in 12 languages, distilled from Minitron-8B
November 20, 2026NVIDIA releases Riva-Translate-4B-Instruct v1.1: a 4B-parameter translation model (BF16, ~8.4 GB) distilled from Mistral-NeMo-Minitron-8B-Base, covering 12 languages and dialects with performance comparable to EuroLLM-9B-Instruct on FLORES and WMT24.
P-Image-OSS: 8-megapixel images in 8 steps, 83 to 101 GB of VRAM to run them locally
November 20, 2026PrunaAI releases P-Image-OSS, an open image generation and editing model based on MiniMax-H3: 8 steps without CFG, 8 MP text-to-image, editing with up to 8 input images and an included prompt upsampler (Qwen3.5-9B). In bf16 it needs ~101 GB of VRAM (83 without the upsampler); the int8/4-bit ComfyUI variant runs in 21 GB.
Parakeet TDT 0.6B v2: local ASR transcription, 24 minutes of audio in one pass
November 20, 2026NVIDIA updates Parakeet TDT 0.6B to v2: the NeMo transcription model (0.6B parameters, English) handles audio up to 24 minutes in a single pass and scores RTFx 3380 on the HF-Open-ASR leaderboard with batch 128. A compact ASR family built for local use.
CompoundingKernel: the agent that writes GPU kernels from scratch, first on all SOL-ExecBench boards
November 20, 2026A self-improving agent system starts from only the PyTorch spec and writes GPU kernels by search: as of Nov 11, 2026 it was #1 on all three public SOL-ExecBench leaderboards (L1, L2, Quantization) on NVIDIA B200, with all 235 kernels correct and 4.40x faster than the baseline.
JunctionBox: Claude Code, Codex and Ollama models under a single control plane, on your hardware
November 20, 2026JunctionBox is a platform to orchestrate coding agents (Claude Code, Codex, Grok) and Ollama models on your own hardware: memory shared across agents, guarded tool use for small models, and local models even without a GPU — the project's tests report BitNet at ~16 tokens/s with under 2 GB of RAM.
Jeveration: System One models generate images in a single pass, 512x512 in 78 ms
November 19, 2026Decision models like Jev use fast parallel prediction: the Jeveration paper shows the same paradigm can generate quality images in a single forward pass, with one checkpoint covering text-to-image, image-to-image and dense prediction. A 512x512 image in 78 ms, up to 122x faster than existing unified models.
Tideway: VRAM and RAM managed as one pool for local MoE inference
November 19, 2026Paper on Hugging Face: TIDEWAY treats system RAM and GPU VRAM as one memory pool, with layouts adapted to the prefill and decode phases. On consumer hardware it claims 1.97x end-to-end speedup and 2.40x on time-to-first-token over the fastest baseline.
Sashiko 0.9.4: the AI code review for the Linux kernel now assesses series design too
November 19, 2026Sashiko, the AI screening tool for Linux kernel patches developed by Google engineers, ships v0.9.4: with 'sashiko review --design' a four-stage experimental pipeline assesses the overall design of a patch series, on top of the per-patch review. First-class e2fsprogs support too; the tool stays LLM-agnostic.
Weight-Space Programming: the model writes the weights by hand, no gradient
November 18, 2026New paper on Hugging Face: three frontier LLMs, with ten labeled examples and a sandbox, write the model's weights by hand instead of training it. Free to design the architecture, all three converge on a sparse table queried by a fixed procedure, with test scores up to 54.4%.
llama.cpp b12199: CUDA 13.5 and ROCm 10.0 builds, sanitized UTF-8 in tool call arguments
November 18, 2026November 8 pre-release with sanitization of invalid UTF-8 in PEG tool call arguments (#31861), Linux and Windows builds with CUDA 13.5 libraries, ROCm 10.0, SYCL and Vulkan, zDNN for s390x, and Snapdragon builds with CPU, Adreno and Hexagon NPU.
WorldFoundry: the open infrastructure for world-model inference and evaluation
November 18, 2026Paper on Hugging Face that defines the generative world model (a predictive dynamical system with persistent state and causally consistent responses) and presents WorldFoundry, open-source infrastructure for reproducible inference and evaluation: a catalog with 270 model entries, 74 benchmarks and 62 registered metrics.
SenseNova-Loop-9B: the looped architecture that turns 32 layers into 56 executions with 9B parameters
November 18, 2026SenseNova's multimodal model reuses the weights of its 24 middle layers twice (looped block with two-stream mHC hyper-connections): context up to 512K tokens and, with 4-bit weights and an 8-bit KV cache, roughly 15 GiB of VRAM for a 256K context. GAIA 69.90, Terminal-Bench 2.1 59.55; on AIDABench it beats Qwen3.8-27B (65.31 vs 65.01).
Envisioning Agentic Robotics: the roadmap for agentic robotics after GPT-6 Astra
November 17, 2026Frontier models with agentic harnesses matched or exceeded task-specific policies within weeks: the paper charts where robotics is heading, with an explicit focus on safety, shared evaluation, and the moment a robot must stop and ask.
llama.cpp b12198: proxy logs downgraded to trace, builds for CUDA 13.5, ROCm 10.0 and SYCL
November 17, 2026The llama.cpp server downgrades per-request proxy logs to the trace level; pre-releases for macOS, Linux (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL), Windows, Android and Snapdragon with NPU. The local server engine keeps its weekly cadence.
byob: the Rust desktop AI agent, no Electron, for the models you already run locally
November 17, 2026byob is a native desktop AI coworker: it reads and writes files, runs code and builds spreadsheets and apps beside the chat, pointed at any OpenAI-compatible endpoint — llama-server, Ollama, LM Studio, vLLM — or Claude. One program for Linux, macOS and Windows, about a 60 MB download, no account.
llama.cpp b12063: device copies for dflash and CUDA 13.4 builds
November 17, 2026dflash copies hidden features via device copies; pre-releases with CUDA 13.4, ROCm 10.0, OpenVINO and SYCL for Linux, macOS, Windows, Android and Snapdragon. The step before b12198 in the engine's release series.
Gamer: miHoYo's vision-language-action foundation model for real-time gaming
November 17, 2026miHoYo releases Gamer, a generalist agent that plays 3D games using only screenshots, keyboard and mouse — no game APIs or privileged state: 10K+ hours of gameplay for pre-training and real-time reinforcement learning.
Local Model Finder: the tool that scans Hugging Face and finds the models that fit your hardware
November 14, 2026An open tool that queries public catalogs and returns the models, with their GGUF quants, that fit the declared RAM or VRAM: one more objective filter before choosing a model.
Duo v1: the full-duplex world model that perceives audio and video while responding
November 14, 2026inclusionAI released Duo v1, a full-duplex Reciprocal World Model: it continuously perceives audio, video and text and responds in real time, with spontaneous conversations, interruptions and sharing of real objects. The weights are on Hugging Face.
Tokspur: the open source inference engine claiming 3x speed on Macs
November 14, 2026Tokspur is an open source inference engine for Apple Silicon built on Metal: claims speak of decode up to three times faster than classic engines on MacBook and Mac Studio. Numbers should be taken with a grain of salt until independent benchmarks exist.
Cloudflare open-sources Kitesurf, its stateless web browser for AI agents
November 14, 2026Kitesurf, the stateless browser built for agents, is now open source: no persistent profile, one session per task, designed for web automation with no state carried between runs.
Octen-WebGR-4B: a 4B open-weight model that memorized 200 million URLs
November 14, 2026Octen-WebGR-4B is a 4-billion-parameter web graph retrieval model in open weights with a live demo: it retrieves existing URLs from natural-language queries, a fresh idea for web resource retrieval without a vector index.
llmtee: the self-hosted LLM proxy that puts agent costs on a spreadsheet
November 12, 2026llmtee is a self-hosted LLM proxy (Go binary + SQLite) that logs every request with tokens, cost, latency and time-to-first-token for OpenAI, Anthropic, OpenRouter, Ollama, vLLM and Groq. It includes daily and monthly per-key budgets with a 429 cap, a dashboard, CSV export and a Prometheus endpoint.
AgentBaton: NVIDIA hands tasks from the capable model to the cheap one, -66% on costs
November 12, 2026The AgentBaton technique (NVIDIA) uses a lightweight detector to hand tasks from a capable model to an efficient one in long-horizon agentic tasks. Average API cost savings of 65.9% and accuracy gains of up to 8.8 percentage points.
Gemma 4 E2B: Google’s on-device-optimized multimodal model, Apache 2.0
November 12, 2026Google releases Gemma 4 E2B, the compact (2.3B effective) variant of the multimodal Gemma 4 family: text and images, 128K context, 140+ languages, Apache 2.0. Built to run on-device and on consumer hardware.
ComfyUI v0.45.0: Qwen 3.x performance, video input and two new image models
November 12, 2026ComfyUI ships v0.45.0: the fourth part of its Qwen 3.x performance work (combined parallel linear weights), video input support for Qwen 3.x, and two new models, RelayVSR and WeVisOne-VL, among many partner nodes in the release.
NEO-VLA: the encoder-free vision-action policy, the model that watches the world in pixels
November 12, 2026NEO-VLA (MMLab, NTU) is an encoder-free vision-language-action framework: pixel images evolve with language across depths and a flow-matching action expert uses representations from multiple layers. Native architectures outperform encoder-based ones.
PyRUA-DataFarm: the coding agent replaces the teleoperator — farm-data policies match teleoperation on five real tasks
November 11, 2026PyRUA-DataFarm: a coding agent replaces the teleoperator — it receives a data request and one calibration episode, rebuilds the task in simulation and renders it as photographic video. On five real-robot tasks, policies trained only on farm data match teleoperation (93 vs 84/100); average cost $21 once + $0.80 per episode, $107 per hour of three-view data.
Open WebUI 0.13.0: tabbed workspace and Desktop/Browser tabs to drive the computer and web apps alongside the model
November 11, 2026Open WebUI 0.13.0 adds the tabbed workspace (files, terminals, previews, voice calls), the Desktop tab to control the connected computer with mouse and keyboard, and the Browser tab for browser use and web app testing, with Chrome mode in a shared session with the model.
llama.cpp b12122: NVFP4 gate/up matmuls and GLU fused in MMQ on CUDA, one kernel for the quantized MLP block
November 11, 2026llama.cpp b12122 fuses NVFP4 gate/up matmuls and GLU in MMQ on CUDA: one kernel for the quantized MLP block, honoring the requested src1 precision and sharing scale checks. The release ships builds for CUDA 12 and 13, ROCm, Vulkan, SYCL and Snapdragon.
NVIDIA releases Isaac GR00T N1.7: the open foundation model family for humanoid robots, 4 variants up to 3B ready for local use
November 11, 2026NVIDIA releases Isaac GR00T N1.7, an open foundation model family (4 variants) for generalized humanoid robot reasoning and skills: multimodal input (language, images, proprioception), flow-matching action transformer, post-training on Bridge, DROID and LIBERO. The 3B model is ready for commercial and non-commercial use.
Flide: the open-source inference engine for decision models, up to 9.5x faster
November 10, 2026OmniJev releases Flide, an open-source inference engine for Jev-like decision models. The technique is exact state forking, which shares computation across multiple questions while preserving decision semantics: the tests claim up to 9.5x versus vLLM and SGLang, with 30+ open-source decision models supported (Decision Index 0.4.1).
llama.cpp v0.9.0: OCP FP8 quantization, unified KV for multi-sequence and Chatterbox TTS
November 10, 2026The local inference engine adds OCP FP8 E4M3 quantization (saturating mode) and the flash_attn_ext_rows op that makes the unified KV cache efficient for multi-sequence inference. New architectures (Clover-35B-A3B-Preview, Limite 1B, Metis-1.6 and Mistral Large 4), Chatterbox TTS, Nemotron 3.5 Transcribe in 40 languages and faster prefill for gemma4 E2B/E4B; ggml moves to v0.29.0 with NCCL multi-node tensor parallelism.
HEELO: agent primitives as reusable building blocks, completed tasks tripled
November 10, 2026Agent-Native Primitives equip tools, code and knowledge with their own LLM-powered experts; HEELO dynamically composes task harnesses from 8 families of primitives. Across 20+ benchmarks and 46 real-world tasks, connecting existing agent systems to HEELO raises completed tasks from 9 to 39 with Claude Code and from 9 to 35 with Codex.
Weight-Space Programming: agents that modify model weights, without training
November 10, 2026A paper evaluated on Hugging Face shows agents inspecting and directly modifying the parameters of a neural network, as an alternative to gradient-based adaptation: 16 editing trajectories across 7 benchmarks and two model scales. Results: uneven cross-task transfer, substantial variation across independent searches, and more limited gains on the larger model.
llama.cpp v0.8.0: the LFM3 family, OpenAI realtime and TTS in the server
November 9, 2026The local inference engine adds the LFM3 family (1.7B text, 2B vision, 2B audio, 2B omni with ASR, TTS and voice chat), an OpenAI-realtime-protocol /v1/realtime endpoint and a dedicated POST /tts, plus new architectures and a hybrid cache for host-resident MoE experts.
Qwen3.8-2.4T-A95B: 2.4T open-weight parameters, 95B active
November 9, 2026Alibaba releases its first Qwen-Max-class model in open weight: 2.4T total parameters with 95B active, a 512-expert MoE (10 routed + 1 shared), Gated DeltaNet hybrid attention, and a native 262k-token context extendable to 1M.
vLLM v0.33.0rc3: CUDA 13.0, new MoE models and speculative decoding on AMD
November 9, 2026927 commits and 389 contributors: vLLM moves to PyTorch 2.14 and Triton 3.8 with CUDA 13.0 only, ABI-stable wheels and an AWS EFA image. Beam-1.0 (501B MoE, 23B active) and Flint (40B, 2.5B active) land, and the XPress refiner with FLy lossy verification adds 1.04-1.31x decode on MI355X and B300.
EmbeddingGemma 2: one space for text, images, video and audio, on consumer hardware
November 9, 2026Google DeepMind releases a 740M-parameter multimodal embedding model: text, code, images, video and audio in a unified 768-dimension space, 100+ languages, 8K context and Matryoshka vectors that cut storage by up to 6x. Apache 2.0, runs on phones and laptops.
Ripple-65B-Preview: ricursiveAI releases a 64.8B dense model under Apache-2.0
November 7, 2026RicursiveAI released Ripple-65B-Preview: a dense model of 64,782,689,360 parameters under Apache-2.0, with a 129.5 GB BF16 checkpoint and quantizations already available. The model climbed Hugging Face's trending chart with 337 likes; it requires custom_code to use. It is a concrete reference point for anyone evaluating a large self-hosted model under a permissive license.
Graph-MIND (now turnbook): local memory for coding agents without a single model call
November 7, 2026Graph-MIND, renamed turnbook, is a local long-term memory for coding agents: it stores and retrieves without a single model call, using word-for-word lookup. It reports 91.8% on LongMemEval starting from 3.1k tokens per query, and is shared across Claude Code, Codex, Cursor, Copilot CLI, Gemini CLI and OpenCode. A concrete alternative to memories that delegate retrieval to the model.
Narro reads web pages and Kindle books aloud with AI voices in the browser
November 7, 2026Narro is a Chrome extension that reads web pages and Kindle books (online reader) aloud using synthetic voices generated locally, in the browser: Kokoro-82M, Supertonic 2 (English, Spanish, French, Portuguese, Korean), Piper and Kitten TTS Nano. In local mode no account and no server are needed; for cloud voices you insert your own API key. No spoken word leaves the machine.
NVIDIA releases Metatron-v1: a humanoid policy that runs in the browser, no server
November 7, 2026Metatron-v1 is NVIDIA's control policy for the Unitree G1 humanoid robot (29 joints), published openly under the NVIDIA Open Model License. The headline: it runs in the browser, on ONNX Runtime Web + WebGPU, with the MuJoCo simulation compiled to WASM. The ONNX model weighs 403 MB, the native PyTorch checkpoint 459,455,983 bytes. No server, no data-center GPU: inference happens on the visitor's own machine.
Qwen3.8-Flash-Next: the 125B MoE with only 6B active parameters, built for agentic work
November 6, 2026Alibaba releases Qwen3.8-Flash-Next: 125B parameters with only 6B activated, plus a 51B n-gram embedding and 4B MTP. The hybrid architecture alternates 48 layers between Gated DeltaNet and Qwen Sparse Attention (512 experts, 10 routed + 1 shared), with a native 262K-token context extensible to 1M. It scores 58.7 on DeepSWE 1.1 — the model is built for the agentic workloads that dominate real usage.
QRAE: compresses DeepSearch agents' context by looking at which tokens actually matter
November 6, 2026DeepSearch agents accumulate ever longer context with each search step, but many retrieved sources don't contribute to the answer. QRAE uses query-relevant attention entropy to compress that context: a new method for cutting tokens and latency in agents that search the web repeatedly.
Nanbeige4.3-33B: 33B MoE with 4 GB of RAM, MIT license, multilingual with Italian support
November 6, 2026Nanbeige4.3-33B is a 33B MoE with 3B active parameters, built for low-RAM devices: quantized, it fits in roughly 4 GB and runs with Ollama on a laptop, Raspberry Pi or Mac mini, with Ollama Cloud and OpenAI-API-compatible clients also supported. MIT license, multilingual — Italian tops the list of 20+ supported languages.
Spark-X2.6-4B: the 4B model from Xhtk that runs locally and goes from 0 to 4.49 on HLE
November 6, 2026Xhtk releases Spark-X2.6-4B in GGUF format under an MIT license: 4B parameters, 3B activated, 48 hybrid layers (36 Sparse Attention + 12 Mamba2), KV cache cut to a third, and context support from 32K up to 256K. In a single training cycle, the HLE (Humanity's Last Exam) score jumps from 0 to 4.49. For anyone who wants a light model that doesn't tax the GPU, it's one of the most complete options right now.
Ollama brings local text-to-speech: 100x faster than real time on Apple Silicon
November 6, 2026Ollama's voice models read out loud with 100-200 ms latency using Apple's Metal: on an NVIDIA GPU with CUDA, speed also exceeds 100x real time. Before a conversation, you can generate the system prompt line as audio, so the user doesn't have to wait on every message. Voice models are OpenAI-API-compatible and work with existing tools (Open WebUI, OpenCode, Claude Code).
Training a specialist LLM on your phone: forward-only optimization with block-spectral coordinates
November 6, 2026User-specific tasks don't need frontier-model capability, but a phone's memory limits block local training. A new method gets a specialist LLM in forward-only mode (no backpropagation) using block-spectral coordinates: data stays on the device, and the model adapts to the tasks that actually matter. A step toward personalized AI that never leaves the phone.
Co-Design Bench: 273 tasks from real user requests to evaluate multimodal coding agents
November 5, 2026A new reference-free benchmark: 273 tasks and 2,331 expert-authored acceptance criteria, curated from 37,463 deduplicated queries across 26,314 real user coding sessions; two tracks (100 multimodal editing tasks, 173 repository editing tasks) and a judge-then-refute protocol that evaluates the running application with browser evidence.
Mistral Large 4: a 1-trillion-parameter MoE, 49B active, released open-weight
November 5, 2026Mistral releases its largest model open-weight: 1 trillion total parameters, ~49B active per token (52B including embeddings and output), natively multimodal, up to 1M-token context, under the Mistral Ecosystem Open-Weight License (a modified MIT — a Model-as-a-Service business needs a separate commercial license).
Gartner: 9 in 10 agentic workflow steps are routine, and open-weight models handle them at a twentieth of the cost
November 5, 2026Per Gartner's report 'Open-Weight Models Will Run 90% of Enterprise Agentic Workflow Steps', 9 of 10 steps in enterprise agentic workflows are routine: open-weight models handle them at parity for a quarter of the cheapest closed model's price — as little as a twentieth versus a frontier model on every step. Planning assumption: by 2030, 90% of operational steps run on open-weight.
Cost-Aware Abstention: on 47,837 IT tickets, a thresholded TF-IDF classifier beats LLMs on cost
November 5, 2026Paper on a public 47,837-ticket, eight-class corpus: under an illustrative cost model (ok 0, human triage 1, wrong routing 5), TF-IDF with logistic regression and a 0.60 threshold is the policy that minimizes cost — the cost model matters more for policy choice than classifier power.
LiquidAI LFM3-1.7B: a 1GB-at-4-bit edge agent model with 256K context and 16 languages
November 5, 2026LiquidAI puts the post-trained LFM3 checkpoint in trending: 1.73B parameters, tri-hybrid architecture, explicit agentic post-training (harnesses like Hermes Agent), 256K-token context, 16 languages including Italian, ~1GB of RAM at 4-bit, LFM Open License v1.0.
NVIDIA releases Cosmos3-Edge-Policy-DROID: a 4B policy for on-device robotics
November 4, 2026The 4B checkpoint of the Cosmos3 family turns natural-language instructions and visual observations from the DROID robot into action trajectories. NVIDIA's edge checkpoint base: OpenMDW 1.1 license, commercial and non-commercial use, released July 2026.
NVIDIA releases Cosmos3-H-Dreams: real-time surgical simulation on a single GPU
November 4, 2026A world model for surgical robotics: from a single endoscopic frame and the stream of commands of both arms, it generates the resulting video in streaming on a single GPU. Derived from Cosmos3-Nano and fine-tuned on clinical data of the Versius platform; NVIDIA Open Model license, commercial use included.
AutoTavern: self-improving LLM agents build a Hearthstone simulator that beats top players
November 4, 2026In 20 days, a system of LLM agents with typed research memory and verification gates built a full Hearthstone Battlegrounds simulator and trained a PPO agent averaging a 1.60 placement in its last 20 games — ahead of the server's ten highest-rated players — answering each decision in 0.03 s. Open-source code.
SynthAPI: scaling laws for tool-use agents with controlled synthetic environments
November 4, 2026A paper studies how synthetic data composition affects the learning of tool-using agents: 458,752 tasks across 849 models in four sizes. At equal budget, diversifying tools and databases beats adding more tasks; agents trained this way outperform baselines on four benchmarks.
TinyAstral: distilling a 180B teacher into 0.8-4B agents for edge robotics
November 3, 2026The paper evolves the harness of a ~180B robot teacher — from 33.4% to 63.6% on RoboTwin — and distills the result into 0.8-4B students running at a fifth of the compute, including closed loop on a real arm. Code, weights and data released.
Ropedia W0: an egocentric world model trained on 10,000 hours of human experience
November 3, 2026A loco-manipulation world foundation model: learns from 10,000 hours of human egocentric video and 3,000 robot hours, and transfers skills across embodiments — robot arms, humanoids, dexterous hands — through one shared kinematic interface. Leading results on five robot benchmarks.
NVIDIA releases Cosmos3-Nano-Sim-Auto: real-time driving simulation with 16B parameters
November 3, 2026Causal-distilled variant of the Cosmos3 family: generates real-time synchronized multi-camera video (up to 11 views) and LiDAR, conditioned on a text prompt and WSM controls, for closed-loop driving simulation. 16B parameters, OpenMDW 1.1 license, ready for commercial use.
Qwen3.5-4B-3D-Coder: 4.5B parameters for 3D code generation, runs locally
November 3, 2026Community fine-tune of Qwen3.5-4B trained to write 3D/Three.js code, built on the 3D-Scenes dataset: Q4_K_M GGUF weights already available for llama.cpp, Ollama and LM Studio. Trending on Hugging Face with 63,000 downloads a month and 495 likes.
MOSS-Image: the OpenMOSS-Team ComfyUI repack, ready in two folders
October 27, 2026Comfy-Org publishes the MOSS-Image repack on Hugging Face with ready-made files: base and SFT in bf16 and int8, color-depth and edit LoRAs, Qwen3.5 9B encoder, and the FLUX2 VAE. Nearly 2.2M downloads in a month.
NVIDIA releases llama-nv-embed-reasoning-3b: 3.2B embeddings with reasoning
October 27, 2026A bidirectional 3.2B embedding model derived from Llama 3.2 3B: it adds a reasoning step before computing the vector, aimed at retrieval and semantic search that also run on 16GB GPUs.
Meet Dessa, the speech model trained from scratch on human conversation
October 24, 2026Block introduces Dessa, a speech model trained from scratch on human conversation. The company already uses it for phone support: a case study in how natural a TTS/STT is when not fine-tuned on a large existing model.
Intel and AMD merge CPU optimizations into oneDNN for PyTorch
October 20, 2026AMD merges its ZenDNN fork into oneDNN and joins the UXL board; Intel brings ACE extensions for matrix multiplication. oneDNN becomes the default x86 library in PyTorch: one software path to optimize for local AI on CPU, regardless of vendor.
OpenMobile-2: (near-)frontier mobile agents with open training environments
October 19, 2026A near-frontier mobile agent is published with fully open training environments and recipes: MobileGym++ simulates 35 commercial apps with cross-app workflows, with controllable reset and verification. Relevant for anyone building on-device agents.
Secret Guard: local LLM gateway that replaces secrets before they reach the provider
October 18, 2026Open-source tool (Rust) acting as a local gateway: it swaps outbound secrets with equal-length realistic mocks and restores them on the way back, so the LLM never touches a real value. One-line integration.
Moose 0.6: secure cloud connections and integrated Ollama in the GNOME app
October 18, 2026Moose 0.6 adds protected credential storage for cloud services, a Local-Only mode with an app-managed Ollama (model download and removal), per-chat or per-provider permissions, a GNOME Files browser to attach documents to conversations, and an expanded local model catalog. 0.6.1 adds Mistral and DeepSeek. Available on Flathub.
llama.cpp b11717: Hadamard for Q1_0 and Q2_0 quantizations on Snapdragon NPU
October 18, 2026Build b11717 adds Hadamard MUL_MAT and GET_ROW operations for Q1_0 and Q2_0 quantizations (flat and tiled) to the Hexagon backend (Snapdragon NPU), and reduces packet counts in the q8_1 quantizer. Qualcomm contribution: a further step toward mobile NPU inference for the most aggressive quantizations.
LLMspeed: test how fast your local model will be before you buy it
October 18, 2026LLMspeed is an open-source tool (Hacker News showcase) that estimates inference speed for 250+ model/hardware combinations: RTX 3060 to H100, Mac Studio M5 Ultra, CPU-only. Measured estimates come from community benchmarks (r/LocalLLaMA, localscore.ai); the rest are derived from similar setups or projected from specs. Useful for choosing hardware before spending.
FLUX 3 Image Klein: Black Forest Labs releases a 5.6B open-weights image model
October 17, 2026FLUX 3 Image Klein is a 5.6B open-weights model for text-to-image generation and editing: one model for prompting, single-reference edits and multi-reference composition, with placement boxes in the prompt. Ships in BF16, FP8 and NVFP4 (5.3GB).
vLLM releases v0.32.0rc3: fixes the Mamba2 chunk scan kernel NaN on sm90
October 17, 2026vLLM's v0.32.0rc3 release candidate fixes the NaN output of the Mamba2 chunk-scan kernel on sm90. Relevant for anyone serving hybrid state-space (Mamba2) models on Blackwell hardware: it's the build that also serves decision models like JEV-35B.
China proposes export controls on 23 AI technologies, weights and agents
October 17, 2026China's commerce ministry proposes adding 40 technologies to its controlled-export catalogue, 23 of them AI: large models, agents and computing clusters. Access via APIs, websites and apps is excluded; already-public open-source models stay out of scope. Public consultation runs until November 15.
llama.cpp b11716: Q1_0 and Q2_0 support on the Snapdragon NPU (Hexagon) backend
October 17, 2026Build b11716 adds Q1_0 and Q2_0 quantization support to the Hexagon backend (Snapdragon NPU), extending mobile NPU inference to the most aggressive quantizations. Pre-release, October 17.
Kyutai STT on Apple Silicon with MLX: streaming speech recognition
October 17, 2026The KyutaiSTT module brings Kyutai's streaming speech recognition to Apple Silicon via MLX. The reference checkpoint is the STT-1B (English and French) quantized to 8-bit, with a causal Mimi encoder producing 32 audio codes every 80ms. CC-BY-4.0 license.
sift: recognizes and summarizes every file with a local LLM, fully offline
October 17, 2026sift is a zero-dependency Python tool that detects a file's real type by content (not just extension), extracts its structure, and asks a local LLM — Ollama, LM Studio or MLX-LM — to classify and summarize it. Nothing leaves the machine; it counts its own numbers and reports files that talk to the assistant.
WorldMemBench: the benchmark for video world-model memory
October 16, 2026A new benchmark measuring how well video generation models keep a coherent memory of the world they generate: previously observed content preserved and coherent evolution over time, alongside visual fidelity and duration.
CLIFT: conformal self-verification for open-source web-agent training
October 16, 2026A training and test-time scaling method for web agents: conformal self-verification instead of frontier-model judges at every step, with a denser learning signal than bare task success or failure.
CALIF: frontier can score full marks without substantive answers
October 15, 2026A paper shows that verifiable-instruction benchmarks can give full marks to answers with little real content, and that using the same criterion as a training reward can induce reward hacking.
DeepMind and Singapore: chat scams blocked with on-device Gemma
October 15, 2026Google DeepMind and GovTech Singapore use lightweight on-device Gemma models to intercept scams in private chats, without sending data to the cloud.
Ling 3.1 Flash: 560B MoE with 25B active, 1M-token context and MIT license
October 15, 2026inclusionAI releases Ling 3.1 Flash: 560B total parameters, 25B active per token, hybrid MoE architecture, up to a 1M-token context window and an MIT license. At 4 bits it fits in 512GB of unified memory — the edge case of today's Mac Studio M5 Ultra article.
From garage to professional maker lab: Raspberry Pi tells it in a free game
October 15, 2026Raspberry Pi launches a free game where you grow a tinker garage into a professional maker lab: the company's story told through progressive upgrades.
MLExpert: a generalist agentic model for autonomous machine learning engineering
October 15, 2026A paper presenting a generalist agentic model that develops and improves machine learning systems autonomously: orchestration of long-horizon experiments, in the direction of AI building AI.
Local Model Guide: which LLMs run on your machine
October 15, 2026An online index collects local models and shows which run on your machine based on available RAM and VRAM: useful before buying hardware for local AI.
NVIDIA releases MagpieTTS Multilingual 357M: local speech synthesis for 12 languages
October 15, 2026NVIDIA has published MagpieTTS Multilingual on Hugging Face, a compact 357M-parameter speech synthesis model covering 12 languages, including Italian and French. Encoder-decoder architecture, designed for low-latency local execution.
ProRubric: fighting reward hacking in rubric-based RL at the protocol level
October 14, 2026In rubric-based reinforcement learning, additive aggregation of criteria lets a policy compensate for one decisive error with advice nobody asked for. ProRubric (ByteDance) groups criteria into protocol-level dimensions that count only when all their criteria hold and no failure clause fires: on clinical consultations, appropriateness rises 10.8 points without losing coverage, with the criteria unchanged word for word.
ClawLoop: self-improving agents without an external teacher model
October 14, 2026ClawLoop (LongCat) trains sandbox LLM agents through a self-sustaining three-phase cycle — task synthesis, SFT/RL training, rollout diagnosis — where the diagnosis feeds the next task synthesis, removing the external model that normally synthesizes tasks and trajectories. With Qwen3.6-27B it improves on SkillsBench, Claw-Eval-Live and ClawGym-Bench; the refined tasks transfer with gains even to Qwen3.5-4B and 9B.
MiniCPM-V 4.7 35B-A3B: the first smooth on-device open-source MoE, 13 tok/s on 3GB of RAM
October 14, 2026OpenBMB releases MiniCPM-V 4.7, 35B-A3B variant: the first open-source MoE able to decode smoothly on a smartphone without a quality compromise — 13+ tokens/second with only 3GB of memory on an iPhone 17 Pro Max. The secret: Lookahead Expert Scheduling on llama.cpp-omni, combined with fine-grained layer-wise quantization and speculative sampling. Day-0 support for Rockchip, Intel, AMD and Arm; Ollama, llama.cpp, vLLM and SGLang.
autotrust/JEV-35B: 35B (3B active) decision model in NVFP4
October 14, 2026Autotrust releases JEV-35B, a 35B-parameter (3B active) MoE decision model fine-tuned on Qwen3.5-35B-A3B for text classification and risk scoring. In NVFP4 quantization it occupies about 25.6GB of VRAM: it runs on a 32GB GPU without going through the frontier.
Kumo-Tabular: foundation model for tabular data
October 14, 2026NVIDIA publishes Kumo-Tabular on Hugging Face, a 500-million-parameter foundation model for tabular data with OpenMDW 1.1 support and ready-to-use checkpoints: an example of local AI on structured business data.
NVIDIA ARDY: interactive real-time human motion with autoregressive diffusion
October 14, 2026ARDY is a 326M-parameter autoregressive diffusion model for interactive real-time humanoid motion generation: online text prompting, flexible kinematic constraints (root paths, keyframes, sparse poses) and sequences up to 8 seconds long. The Core-RP-20FPS-Horizon40 checkpoint, trained on the Bones Rigplay 1 dataset, is on Hugging Face since October 7 and optimized for NVIDIA Ampere, Hopper and Blackwell GPUs.
ComfyUI v0.40.0: comfy_attention for all models, local asset API and 3D outputs
October 14, 2026v0.40.0 extends comfy_attention and AttentionTensorContainer to the rest of the models, brings the asset export API directly into the local server, and makes every 3D node output its models as a standard 3D item. Plus: CreateCameraInfo node with a CameraInfoState widget, video metadata extracted into system_metadata on ingest, and the CSP updated for blob: and data:.
Linux kernel plans to drop the «Assisted-by» tag for AI-written patches
October 14, 2026Now that AI assistants are part of daily use, Linux kernel patches with AI/LLM contributions had to declare it with the «Assisted-by:» tag. With this tooling now ubiquitous, kernel developers are looking at dropping the requirement — the declaration becomes a choice, not a mandate.
Relevance does not imply applicability: experience activation for personal GUI agents
October 13, 2026Personal GUI agents lean on interaction history to infer intent; the paper finds that task-relevant experience is largely 'spent' early and does not help every decision within the task.
T3lescope: high-fidelity generative 3D surface reconstruction from images
October 13, 2026The paper reconstructs high-fidelity 3D scene meshes from posed multi-view images without per-scene optimization, from single objects to large outdoor scenes.
Scaling Agentic Data for Long-Horizon Terminal Intelligence
October 13, 2026A new Hugging Face paper shows how terminal agents are moving beyond local operations toward real professional tasks - accumulating evidence, adjusting strategy, and turning intermediate results into final deliverables.
ARCS: precise text-to-SQL via structured disambiguation
October 13, 2026Ambiguous user questions are a primary source of text-to-SQL errors; the paper addresses them with structured disambiguation instead of costly conversational clarification.
EpochGo: local-first multi-agent AutoML on Ollama, zero API cost
October 12, 2026A GitHub project running multi-agent AutoML workflows entirely on Ollama - no API calls, no cloud. Verified against the primary source; early-stage (low HN signal).
VisionWeave: elastic visual representations as a native capability of multimodal models
October 12, 2026Multimodal models pay a high cost by encoding every image into fixed dense tokens. VisionWeave proposes elastic visual representations: fine detail where needed, compact encoding elsewhere, without sacrificing understanding.
InstantFusion: a shared latent interface for heterogeneous image generators
October 12, 2026New paper on Hugging Face: aligning the latent representations of image generators with different architectures (diffusion and flow matching) is still an open problem. InstantFusion proposes a shared latent interface to make generators interoperable.
Ubuntu's Myna speech-to-text is now available in tech preview
October 12, 2026Canonical's Myna speech-to-text, part of the context-aware desktop effort for Ubuntu 26.10, is now available as a tech preview - also on LTS releases.
Open WebUI 0.12.0: realtime voice calls
October 10, 2026The new Open WebUI release adds a Realtime mode for voice calls: a voice model handles the conversation and hands real questions to the selected chat model, which answers using its history and tools. It can be enabled by administrators in the audio settings.
Tutor: a self-hosted LLM tutor that teaches your books and docs
October 10, 2026Open-source project on GitHub: a local tutor that studies the books and documents you give it and teaches the content with quizzes and questions, using a local LLM. Designed for people who want to learn without sending their materials to a cloud.
Linux 7.4: laptop and gaming handheld support improvements incoming
October 10, 2026The Linux 7.4 kernel merge window is receiving a batch of x86 platform driver improvements for laptops and gaming handhelds: many driver branches have been queued ahead of the next kernel version.
KNOD: in-kernel GPU offloading for network packet processing
October 10, 2026The open-source KNOD project lets AMD GPUs handle network packet processing without specialized hardware, with high performance and low power draw. A development update was shared at the Linux Plumbers Conference in Prague.
Marformer: predicting missing-data distributions to decide better
October 9, 2026A transformer trained to predict the conditional marginals of missing variables, the key ingredient for computing Bayes risk and the value of one more observation before deciding under incomplete information.
An 11.4 MB open-weight decision model: the System-1 for devices
October 9, 2026Hugging Face hosts ej, a small 11.4 MB calibrated decision model with open weights: given a state and typed questions, it returns one probability distribution per question in a single pass, without decoding tokens. Designed to run on CPU with a 128 MB resident-memory peak.
OnTrack: real-time monitoring and intervention on LLM agent trajectories
October 9, 2026A method to monitor and intervene in real time on LLM agent trajectories, using streaming structure-aware optimal transport: it avoids the cost and latency of a per-step safeguard agent while staying proactive rather than examining logs after the fact.
Memento 3: LLM agents that learn explicit world-model rulebooks
October 9, 2026Memento 3, the continuation of the Memento series, lets frozen-LLM agents continually learn explicit world models in unfamiliar environments, using a reflective rulebook that updates as new evidence arrives.
SpaceFlow: training-free, locally controllable 3D generation
October 9, 2026A training-free pipeline for locally controllable 3D generation: each geometric primitive acts as a proxy for an object part and is assigned its own independent local control, overcoming the limits of methods built on a single global control strength.
SpatialOPSD: spatial intelligence learned through self-distillation
October 9, 2026A new paper shows how a multimodal model can internalize the spatial reasoning ability of a tool-using agent, cutting inference cost: the model learns from verified traces of its own executions, with no reliance on external tools.
reViT: a single recurrent transformer block matches a full-depth encoder
October 9, 2026A new paper shows a single Transformer block, applied recurrently, matches a full-depth vision encoder's accuracy at comparable inference FLOPs. The key is representing the FFN at each recurrent depth as a convex combination of a small shared expert bank, with no intermediate feature distillation.
Boro: NVIDIA brings local AI to Linux kernel development
October 9, 2026NVIDIA is developing Boro, a Rust CLI that brings local AI to Linux kernel development: patch validation, builds, boot and tests, built for backporting. Led by Andrea Righi and presented at LPC 2026, it is Apache 2.0 and runs on local back-ends (Claude, OpenCode, Codex) or OpenAI-compatible servers.
AI recreates Photoshop and Premiere in Rust, open source
October 8, 2026The ArtCraft project released Photocraft, Filmcraft and Lightcraft: Rust reimplementations of Photoshop, Premiere and Lightroom written with AI agents, available as native binaries for Linux, macOS and Windows. An interesting demonstration of what agents can do, with open legal questions.
Google releases ML Drift, an open-source on-device GPU inference engine
October 8, 2026The Google AI Edge team released ML Drift, a cross-platform GPU compute engine for on-device inference under the Apache 2.0 license. It covers OpenGL ES, OpenCL, Metal and WebGPU, supports 5D tensors for 3D networks and generative models, and is the new acceleration core of LiteRT.
RunningTab: direct workspace interaction with file tabs for AI agents
October 8, 2026A new Hugging Face paper proposes that LLM agents work directly on workspace files through dedicated tabs, with no indexing: the agent searches and reads documents from a terminal and produces deliverables from multiple files in one flow.
The Jev architecture was open-sourced a year ago: model, dataset and paper included
October 7, 2026On LocalLLaMA, one author describes building the architecture behind the Jev decision models a year ago and open-sourcing it completely: model, dataset and paper. Backstory that accompanies the arrival of the System One format in llama.cpp and the opening of the decision-model ecosystem to the community.
llama.cpp adds decision models via the /v1/systemone endpoint
October 7, 2026The llama.cpp server now supports decision models: send a state (text, JSON or a screenshot) plus typed questions, and the model returns a probability for each option in a single forward pass, without generating text. Three question types (choice, score, noul) and six models from 144M (Julia-1) to 27B (OpenJev, Clef), answering in 3-43 ms on an RTX PRO 6000, Apache 2.0 except OpenJev (CC BY-NC).
Aleph Alpha releases Kolibri: 78B parameters, 1M context, built in Europe
October 7, 2026Aleph Alpha's new sovereign open-weight model, trained from scratch in Germany: 78B parameters (3.46B active) with up to 1M tokens of context. Built for mission-critical environments - multi-step reasoning, structured extraction, RAG and reliable tool calling, with native German and English. Weights are on Hugging Face, with a full technical report.
Hugging Face research: tool-using agents rarely turn their own evidence judgments into stopping decisions
October 7, 2026A study of tool-using agents in a retrieval environment with controlled source failures: when a tool keeps returning useless results, the agent judges them as useless but rarely turns that judgment into a stopping decision. The contrast between stopping after long and short runs of judged-useless results is zero for clock- or deadline-driven stopping — it measures the gap that separates today's models.
Attacca: goal-directed control under state continuity for long-horizon embodied agents
October 7, 2026Visual goal-conditioned policies for embodied agents are almost always evaluated on isolated interactions where the target is already visible. Attacca tackles the real setting: sequences of interdependent tasks in continuous long-horizon execution, where each task starts from the state left by the previous one and the target isn't necessarily in scene. A step toward agents that accomplish complex objectives in one uninterrupted session.
GUI-HARVEST: self-improving GUI agents through evidence-driven harness evolution
October 7, 2026The executable harness around a GUI model — how observations are assembled, actions executed, and verification, recovery and termination controlled — determines most of an agent's behavior. GUI-HARVEST optimizes it automatically from execution evidence, tackling reconciling model intent with observed visual effects, diagnosing failures under variable execution, and controlling termination.
Kandinsky 6.0 Video: foundation models for synchronized video and audio generation
October 6, 2026Kandinsky Lab publishes the Kandinsky 6.0 Video paper: foundation models for synchronized video and audio generation, the day's #1 paper on Hugging Face with 8 models and 6 Spaces already linked.
GridCore: a scheduler that lets several local LLMs share one GPU
October 6, 2026An open-source tool that schedules several local LLMs on the same GPU, keeping them from fighting over VRAM in turns. Useful for people running multiple models (an assistant, an embedder, a classifier) on one card who want predictable latencies.
PaLoRA: Paced Low-Rank Adaptation for Continual Learning
October 6, 2026The PaLoRA paper shows that the step-size restriction in LoRA-based continual learning should grow with the effective rank of accumulated knowledge, and proposes an adaptive pacing scheme gaining 4% accuracy on 50-task benchmarks.
LTX-2.5: synchronized video and audio locally, 22B parameters and native multishot
October 5, 2026Lightricks releases LTX-2.5, open weights: video generation with synchronized audio from text, image, or video, with native multishot that holds identity and style across scenes, a new diffusion video decoder replacing the VAE stage, and a Gemma 4 12B text encoder. The weights are built for running and fine-tuning on your own infrastructure.
TalkScribe: 100% local on-device dictation on macOS with Whisper and a local LLM
October 5, 2026A native macOS app turns dictation into text with Whisper running on-device plus a local LLM (Qwen) rewrite pass on the M-series GPU: no byte leaves the computer, no cloud account. Audio goes through Whisper, then the local model cleans up punctuation and formatting before delivering text to the target app.
Backburner: iPhone offloading for faster local LLMs
October 3, 2026On GitHub, an experimental project offloads part of local LLM inference to a paired iPhone, using the phone's GPU and memory as a secondary accelerator — an idea for anyone wanting more tokens per second without swapping graphics cards.
Moose: the GNOME local-AI app runs on Ollama
October 3, 2026Moose is a new native GTK4/libadwaita app for GNOME, written in Rust under GPLv3, that lets you chat with local models via Ollama: it handles Ollama setup, model management and loading local documents and images. It's on Flathub, the code is on GitHub, and it was announced on This Week in GNOME.
Agent Priors: policy learning guided by agent model priors
October 2, 2026A Hugging Face community paper explores how the implicit priors of agent models can guide policy learning, reducing the cost of training LLM agents in structured domains.
llama.cpp b11337: the October release run widens the CPU/edge backend matrix
October 2, 2026llama.cpp b11337 widens the CPU/edge backend matrix: Snapdragon (Adreno and Hexagon), OpenVINO, SYCL FP16/FP32, CUDA 13, Android, and ROCm 10, plus server fixes. It is the eighth release in the October 2 batch.
funcroute: a router that gives a text-only LLM model eyes and ears
October 2, 2026A small open-source router (v0.4.0) that adds multimodal capabilities to a text-only LLM model: images and audio are handled at the routing layer, so the local model stays the one you already run, without swapping it out.
Intent drift in LLM agents: measuring and repairing it
October 2, 2026New research paper: when a user changes their mind during a multi-turn conversation, superseded parts of the intent keep influencing the final answer (intent drift). The paper introduces IntentFlux, an executable benchmark that converts verifiable tasks into dialogues with controlled intent changes, and proposes methods to detect and repair the failure mode.
Sriti Core: a local-first LLM router that cascades from Ollama to the cloud
October 1, 2026An open-source routing proxy (Apache 2.0) that cascades LLM requests: it starts with the local model on Ollama and climbs to the cloud only when needed, with semantic caching and per-hop reliability tracking. An interesting pattern for those who want local by default.
Ordinal-scale bias in JEV-like direct-decision models
October 1, 2026Research paper on direct-decision models: JEV 1.13 and three open KEV models do not use the ordinal scale supplied by the user faithfully. On ANLI, JEV assigns 38.8% of all predictions to the top option regardless of content — more choices, fewer decisions.
TACO: ternary optimizer that cuts fine-tuning memory by 174x
October 1, 2026University of Central Florida researchers propose TACO, a ternary column-wise one-sparse optimizer for LLM fine-tuning: it keeps first-order gradients while shrinking persistent optimizer state from 27.7 GB to 0.16 GB on OPT-13B.
Local model won't stop talking? The fault is often the chat template
October 1, 2026A well-argued article explains why a local model that stalls, ignores the system prompt or keeps generating past its answer is almost always a chat-template or token-metadata problem in the GGUF file, not a weights problem — and how to fix it with gguf-surgeon without re-quantizing.
Cloudflare releases clef: 27B decision model, Apache 2.0
October 1, 2026Cloudflare made clef public, a 27B model (base Qwen3.8-27B) that turns a state — text, JSON, images, or video — and a schema of typed questions into structured decisions: one probability per option in a single pass, with no text generation. Jev/SystemOne API compatible; a smaller, faster variant clef-flash is available too.
vLLM v0.31.0rc3: randomized dummy inputs support in Model Runner V2
October 1, 2026The v0.31.0 release candidate for vLLM adds support for randomized dummy inputs in the new Model Runner V2 (PR #58411), a step toward more robust testing and profiling of the new execution engine.
From 17 hours of video to a second brain: local transcription with Whisper on MLX
September 30, 2026A well-documented experiment: 67 course videos transcribed locally and offline with whisper-large-v3-turbo rebuilt on MLX for the Mac's GPU — 17 hours of video processed in 34 minutes, without a single byte leaving the machine.
Qt 6.12 LTS released with QML hot reloading
September 30, 2026Qt 6.12 LTS arrives with QML hot reloading in experimental form and the new StyleKit: faster development tools for desktop and embedded UI builders — relevant for local UIs of apps with built-in AI as well.
vLLM ships proto-v0.4.0: the next step for the inference engine
September 30, 2026The vLLM project tags proto-v0.4.0, the experimental development track that precedes the stable releases of the language-model inference engine. One more step for anyone serving local models in production.
Ollama 0.35.0: Jev-style decision models via the /v1/systemone endpoint
September 30, 2026With Ollama 0.35 you can use decision models based on TypeSafe's Jev API: send text as a state with a set of named questions and a local model answers them all in one request, with a probability per option and no free-form text generation. Built for ticket triage, model routing, and content moderation.
WUSH-KV: KV cache quantization with data-adaptive transforms
September 30, 2026A new method, WUSH-KV, cuts the memory and bandwidth the KV cache takes up in long-context inference by quantizing it at low precision with data-calibrated transforms. At 2 bits, the approach matches or beats the OSCAR transform across all tested models and tasks. It's a sign that memory optimization is no longer only about the weights, but also the context memory.
Ollama 0.35 brings decision models: nimble, a 9B model for typed classification
September 30, 2026Ollama 0.35 adds decision-model support: Bespoke Labs' 9B nimble model runs entirely locally and answers typed questions (true/false, numeric scale, multiple choice) through the new systemone API.
Magnitude: an open-source inference engine that optimizes itself
September 30, 2026Launched on Hacker News (YC S25), Magnitude is a Rust inference engine that compiles and tunes its kernels directly on your target hardware. On open models it claims up to 2× the speed of llama.cpp, on Apple Silicon, NVIDIA, AMD or CPU. Verified against the official repository.
Jev in the Wild: data-driven analysis of Jev decision models in the wild
September 28, 2026CUHK study of Jev decision models: 2,050 real-world use cases analyzed, from triage and routing to moderation and labeling. The paper maps how Jev models work, their applications, and the ecosystem of models that answer with structured labels instead of free text.
BenchLM publishes its September ranking of the best local LLMs: VRAM defines the tiers
September 27, 2026The September 2026 ranking orders open-weight models by VRAM tier: GPT-OSS 20B and ZAYA1-8B lead the 8-16 GB tier, Qwen3.6-27B (46.49) and Gemma 4 31B (44.84) dominate the 16-24 GB tier on a single RTX 4090 at Q4, and trillion-parameter MoEs need 8x H100. A useful reference when choosing a model size for your hardware.
Qwen3Guard-Stream-0.6B: real-time safety moderation for local AI
September 27, 2026Qwen3Guard-Stream is a 0.6B moderation model with a token-level classifier for real-time monitoring during incremental generation. Three severity tiers (safe, controversial, unsafe) and 119 languages: a small, local component to filter outputs without sending them to the cloud.
A wall that listens: the local-LLM pipeline behind an AI party in Vilnius
September 27, 2026A creative experiment in Vilnius: a wall with microphones and speakers powered by a fully local LLM pipeline. Participants talk to the wall and get responses generated on site — no cloud, no external servers.
DeepSeek releases V4.1 Flash: 552B MoE with 1M context and MIT license
September 26, 2026DeepSeek V4.1 Flash pushes the MoE architecture further: 552B total parameters but only 8-16B active per token, a 1M-token context window, and an MIT license. A model built for online serving, but useful as a reference for learning to read model spec sheets.
AMDXDNA: Linux driver enhancements for Ryzen AI 300 NPU3
September 26, 2026A series of 21 patches for the open-source AMDXDNA driver extends NPU3 (Ryzen AI 300) support: kernel-mode command submission, full suspend/resume handling and telemetry, paving the way for NPU-based local AI on Linux.
ComfyUI v0.37.4: stability patch for local generation workflows
September 25, 2026ComfyUI reaches v0.37.4, a patch in the 0.37.x series following the native Qwen-Image 2.1 support introduced in v0.37.0. If you use ComfyUI for local image generation, this is the build to run right now.
Google releases Gemma 4 under Apache 2.0 with context up to 256K
September 25, 2026Gemma 4, released in March 2026, marks a license change for Google's family: for the first time the weights are under Apache 2.0. Available in E2B, E4B, 26B A4B (MoE) and 31B sizes, with context windows up to 256K tokens and text, image, and audio input. The 12B Unified followed in June — an ideal size for PCs with consumer GPUs.
PISA: block sparse attention with log-linear complexity
September 25, 2026The PISA paper proposes a pyramid Top-K selection strategy that brings block sparse attention to O(N log N) complexity, cutting the quadratic bottleneck of block selection; Triton kernels for training and inference.
Bonsai-2 27B: 27B ternary reasoning that runs on a laptop
September 25, 2026Prism ML releases Bonsai-2 27B in GGUF: a 27B reasoning model with ternary weights at 1.72 bits per weight, 5.95 GB and a 262K-token context, retaining 98.2% of the FP16 intelligence and reaching ~47 tok/s on an Apple M5 Max laptop.
Open WebUI v0.11.4: per-file metadata and faster hybrid search on large knowledge bases
September 24, 2026Open WebUI release v0.11.4, published on September 21, brings two direct RAG improvements: metadata attached to uploaded files now travels with each chunk and arrives with the retrieved sources, and hybrid search now runs natively in the database (pgvector) instead of loading the whole collection into memory, making large knowledge bases dramatically faster. The slim image also drops to around 175 MB.
Edge0 releases Audio8-ASR-Infinite: real-time streaming speech-to-text
September 24, 2026A new open speech-to-text model: real-time streaming with low latency, built for live dictation and captions. One more step toward audio that genuinely runs locally, without going through the cloud.
Ollama v0.34.4: faster structured outputs and faster Qwen on Apple Silicon
September 23, 2026Ollama's v0.34.4 release applies structured outputs on thinking models in a single pass (faster and more reliable), fixes 'model not found' errors with large local libraries, and speeds up Qwen 3.8 prompt processing on Apple Silicon. A useful update if you run Ollama in production.
Needle 2: a 14MB model that turns text into actions on a Raspberry Pi 5
September 22, 2026Needle 2, from Cactus Compute, is a 14MB function-calling model that runs on the Raspberry Pi 5 CPU, with no AI HAT: 'turn on the LED' becomes an action in 78 milliseconds. It is not a chatbot: it is trained for one task only, reliable structured on-device actions. Needle 3 is already out.
Mistral and Mozilla bring open, private and multilingual AI to the browser with Firefox Smart Window
September 22, 2026Mozilla and Mistral announced a partnership integrating Mistral's AI — open, multilingual models with a privacy focus — into Firefox Smart Window, the browser's AI panel. A signal in the direction of today's article: bringing AI into everyday work tools, with data staying under the user's control as much as possible.
NVIDIA NemotronLabs VoiceChat 11B: real-time open voice conversation with tool calling
September 22, 2026NemotronLabs VoiceChat 11B is an end-to-end full-duplex voice model: audio understanding and generation in a single architecture, without the cascaded ASR→LLM→TTS stack. It is the first open full-duplex model to support tool calling while keeping the natural flow: ~450ms responses, interruptions, and natural turns. The code is on GitHub.
llama.cpp: weekly report Sept 7–14 — build b4991, Gemma 4 thinking bug fixed and Vulkan r615 driver for CI
September 21, 2026The weekly report on the llama.cpp project covers the current build b4991 and the week's activity: a bug where Gemma 4 in thinking mode emitted progressively longer trailing garbage was resolved by trimming whitespace in the chat template; the Vulkan CI runners moved to the NVIDIA r615 driver, and the report analyzes the community's most active issues.
Qwen releases Qwen-Image 2.1: open image generation and editing that runs locally
September 20, 2026Qwen-Image 2.1 is a 7B open-weight model (32-layer DiT) that unifies text-to-image generation up to 2048x2048, editing with up to 10 reference images, and native RGBA transparency. With ~14GB of VRAM in bfloat16 it runs on consumer hardware, and ComfyUI supports it natively: a concrete step for cloud-free image production.
vLLM releases version 0.29.0: Model Runner V2 by default and support for new models
September 20, 2026The vLLM project released version 0.29.0, with 594 commits from 277 contributors. The main highlight is Model Runner V2, now the default for all models, with CUDA graph memory profiling for KV cache auto-sizing. New model support also lands: Qwen3.8-Flash-Next, Hy4-preview (a 770B MoE with native MTP), Granite SWA, and Kimi K3 checkpoints in NVFP4.
Qwen releases Qwen3.8-Omni-Flash: an omnimodal model with a 1M context
September 19, 2026Alibaba's Qwen team released Qwen3.8-Omni-Flash, a native omnimodal model that jointly handles text, images, audio and video with a one-million-token context window. It is built around agentic capabilities: it plans tasks, calls tools and produces finished audio-visual content. The Qwen family keeps growing right as this guide analyzes it for local use.
Open WebUI v0.11.1: tool approvals, ask_user, and faster streams
September 18, 2026Open WebUI v0.11.1 adds tool-call approvals (a pause button between the model and its tools), a built-in ask_user function that lets the model pause and ask you a question mid-answer, a terminal file browser with in-place document previews, and delta streaming that cuts reply traffic by roughly 99%. The full release includes more than ninety additions and over two hundred fixes.
Ollama v0.34.0: Ollama models in ChatGPT Desktop
September 5, 2026Ollama v0.34.0 lets you use local Ollama models directly in ChatGPT Desktop on macOS, improves structured output performance on Apple Silicon, and adds OpenAI-compatible client tool search and response compaction.
Ollama v0.33.3: Gemma4 supports images and audio on MLX
September 2, 2026Gemma4 can now process images and audio on the MLX engine. Also, cached prompt tokens are now reported correctly and default parameters defined in GGUF files are honored.
Ollama v0.33.2: dark mode returns in the app and the Claude Desktop proxy no longer interrupts requests
August 27, 2026The Ollama app again follows the system appearance (dark mode restored), an already-running macOS instance is no longer duplicated, and the Claude Desktop proxy no longer interrupts in-flight requests when the model catalog updates.
Ollama v0.33.1: Qwen3.8 Flash Next support and structured output on MLX
August 26, 2026New Qwen3.8 Flash Next support on the MLX engine, structured output for mlxrunner, and a fix for Metal GPU timeouts when loading models from slow storage.
Ollama v0.33.0: Claude Desktop can use local models as a gateway
August 21, 2026The release adds Claude Desktop setup with Ollama as a third-party gateway provider, plus major caching improvements: prefill restore points make retries far more efficient and fix a hang when agent clients cancel long prefills.
Meta announces Llama 4: the new 100B parameter open source model
July 27, 2026Meta released Llama 4, the new open source language model with 100 billion parameters. Performance surpasses GPT-4 on many benchmarks and the model is optimized for local execution with quantization.
Apple integrates MLX 0.20 with native support for quantized models
July 25, 2026Apple updated the MLX framework with native support for 4-bit quantized GGUF models, allowing larger models to run on M-series chips.
Ollama adds new models and improves performance
July 24, 2026The Ollama release adds support for new open-source models and measurable inference performance improvements. Tip: after every update, restart the server and verify your models with `ollama list`.

