Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active

Qwen3.8-2.4T-A95B is the first Qwen-Max-class model in open weight: 2.4 trillion parameters in total, 95 billion active per token. The news matters to people who run models locally, but not for the reason you'd think: this is not about downloading it onto a workstation. It's about understanding what it weighs, what it costs, and what you get. The math is below.
What 2.4T actually means
"2.4 trillion" is not a number you can feel: it's a datacenter scale. The concrete facts, from the model card:
- MoE with 512 experts: per token, 10 routed + 1 shared expert activate, for 95B active parameters. The rest of the model is there, but it isn't computed.
- 92 layers, hidden dimension 8192, with multi-token prediction (MTP) trained over multiple steps.
- Hybrid attention: 3 of every 4 layers use Gated DeltaNet (linear attention, constant state), 1 of 4 uses classic attention (Gated Attention). It's the same family of tricks that keeps the KV cache small: on a long context, cache memory only grows for the 23 classic-attention layers, not all 92.
- Context: 262,144 tokens natively, extendable to ~1M.
- Formats: BF16 weights in the repo, with a separate FP8 variant. The official API build (Qwen3.8-Max) adds vision input and 1M context by default, per the official blog.
The point that matters for the math: the bytes that must live in memory are the 2.4T in full, not the 95B active. In an MoE the experts sit in memory and each token touches a few of them: the machine has to be able to hold all of them.
How many bytes that is
Parameters × bytes per parameter. With the standard GGUF quant ratios (the same ones we use for normal-sized models):
| Format | Bytes/param | Weights on disk |
|---|---|---|
| BF16 (as released) | 2.0 | ~4.8 TB |
| FP8 (separate variant) | 1.0 | ~2.4 TB |
| Q8 (GGUF) | ~1.1 | ~2.6 TB |
| Q4_K_M (GGUF) | ~0.55 | ~1.3 TB |
| Q2_K (GGUF) | ~0.38 | ~0.9 TB |
Even at the most aggressive quant in the table the weights clear 900 GB. The question stops being "which quant" and becomes "which machine".
Where it fits
The uncomfortable answer: no single consumer node. The top of the market today:
| Machine | Memory | Fits Q4_K_M (~1.3 TB)? |
|---|---|---|
| NVIDIA DGX Spark (GB10) | 128 GB | No |
| AMD Ryzen AI Max+ 395 (Strix Halo) | 128 GB | No |
| Mac Studio (M5 Max) | 128 GB | No |
| Mac Studio (M5 Ultra) | 512 GB | No — 512 < 1,300 |
The consumer record, 512 GB of unified memory, covers barely a third of the model at Q4. If you want a single node that fits with headroom, the next step up is server memory: 1.5-2 TB of ECC RAM on a multi-socket workstation, or GPUs in tensor/parallel-expert configurations.
The realistic consumer alternative is a multi-node setup:
| Configuration | Total memory | Q4_K_M weights | Hardware cost |
|---|---|---|---|
| 4× Mac Studio M5 Ultra 512 GB | 2 TB | Fits, with room for the KV cache | 4 × 6,699 = 26,796 € |
| 8× H100 80 GB (SXM) | 640 GB | No, even at Q2_K | 8 × 30,000 = 240,000 € |
| 24× H100 80 GB (SXM) | 1.9 TB | Fits | 24 × 30,000 = 720,000 € |
The 8× H100 row is in there to show the gap: 640 GB is not enough for any quant in the table above. You need multiples more, and the price follows.
The running costs: amortization and power
Same assumptions as our other machine math: 36-month amortization, 24/7 usage at a 50% average load, electricity at €0.25/kWh.
4× Mac Studio M5 Ultra 512 GB (480 W peak per unit, ~240 W average at half load):
- Amortization: 26,796 € ÷ 36 = ~744 €/month
- Power: 0.24 kW × 730 h = 175 kWh per Mac per month → 4 Macs = 701 kWh → × 0.25 = ~175 €/month
- Total: ~920 €/month, all included
24× H100 80 GB (700 W peak per card, ~350 W average):
- Amortization: 720,000 € ÷ 36 = 20,000 €/month
- Power: 8.4 kW average × 730 h = 6,132 kWh → × 0.25 = ~1,530 €/month
- Total: ~21,500 €/month, before room, cooling, and networking
The H100 fleet's electricity bill (€1,530/month) costs more than the entire Mac fleet's monthly total, which spends €175. On big MoEs, the architecture difference between unified memory and HBM shows up on the meter before it shows up in benchmarks.
What speed you'd get
On a single node, the rule we use for MoEs applies here: decoding is bandwidth-limited, and you divide bandwidth by the bytes of the active experts. 95B active at Q4_K_M ≈ 52 GB:
| Bandwidth | Theoretical tok/s (1,200 ÷ 52) |
|---|---|
| Mac Studio M5 Ultra, 1,200 GB/s | ~23 |
| DGX Spark, 273 GB/s | ~5 |
Careful: the 512 GB Mac does not hold the model, so the 23 tok/s is a counterfactual — the speed of a machine that doesn't exist. In multi-node the math changes: each token crosses all 92 MoE blocks, and at each block the activations make an all-to-all round between the nodes hosting the experts. The bottleneck is no longer memory bandwidth but the interconnect: Thunderbolt (10-15 GB/s) or 25/40GbE. There are no public numbers for this exact configuration, and we won't invent any: the order of magnitude is set by the interconnect, and if you need an always-on server with SLOs, start from the assumption that a consumer multi-node build is a project to validate, not a purchase.
When it makes sense
- If today you run a Qwen3.8-27B or a 30B-A3B on 24 GB of VRAM: for the vast majority of uses — RAG, assisted coding, automation — those are enough. This article is not an incentive to buy something else, it's the cost map for the case where you genuinely want Max-class quality without an API.
- It makes sense locally when the data cannot leave, the model must be available 24/7 without a token meter, and you already have space for a rack or a table full of Macs. In that case, the Mac fleet at ~920 €/month is the number to compare against your API token volume.
- It does not make sense for someone starting from zero: a single 512 GB Mac with a 30-70B model gives you a complete local AI for a quarter of the price, with half the logistical problems.
In summary
Qwen3.8-2.4T-A95B brings the Qwen-Max class to open weight: 2.4T parameters, 95B active, context from 262k to 1M. In bytes, ~1.3 TB at Q4_K_M — more than any consumer node (max 512 GB) — so local for this model is necessarily multi-node: 4 Mac Studio M5 Ultra at ~26,800 € (~920 €/month all included) or 24 H100 at 720,000 € (~21,500 €/month). Multi-node speed is set by the interconnect, not by memory. If you don't have infrastructure yet, the right model for you is smaller: the open frontier has reached Max-class, but your hardware probably hasn't.
Article generated with AI
In these reading paths
Enjoyed this article?
Subscribe to the newsletter for new tutorials and guides.
Related Articles

Ripple-65B: a 64.8B dense in Apache-2.0, the math of running it locally
64.8B parameters, Apache 2.0, 1M of context and an NVFP4 checkpoint designed for 128 GB of unified memory: the math and the criteria for running a frontier dense locally.

QRAE: compressing DeepSearch agent context to save tokens and latency
QRAE uses the model's own attention entropy to decide which retrieved chunks stay in the context: tokens -45%, latency -30% and higher success across four benchmarks.

