gpt-oss: OpenAI's open reasoning model

OpenAI has released its first open-weight reasoning models: gpt-oss-20b and gpt-oss-120b. The first one fits in 16GB of memory and behaves like a 24GB model. Here is what that means in practice.

What gpt-oss is and why it matters

GPT-OSS is OpenAI's open-weight model family, released under the Apache 2.0 license: downloadable weights, trainable, with no usage restrictions. It is OpenAI's return to the open front since GPT-2, and it comes with a rare trait for the consumer tier: it is a reasoning model, trained with the same approach (SFT + high-compute RL) used for proprietary models like o4-mini.

The version that matters for consumer hardware is gpt-oss-20b: 21B total parameters, but only 3.6B active per token.

MoE: 21B total, 3.6B in action

gpt-oss is a Mixture-of-Experts. In practice the model has 32 "experts" (groups of parameters) and activates only 4 per token: those 3.6B are the ones that actually work. The double benefit:

• Less active memory: generation speed depends on active parameters, not total. 3.6B active means small-model speed.
• More capacity: the 21B total stay available, so reasoning quality stays at big-model level.

This is the same architecture Mixtral and the Qwen3 MoE family use: the difference here is that the weights are already MXFP4, which changes the hardware picture quite a bit.

MXFP4: why you do not need to quantize

MXFP4 is a 4-bit floating-point format with block scaling: every group of 32 weights carries its own scale factor. It is different from the GGUF integer quantizations (Q4_K_M and friends) and it is the format OpenAI trained the model in directly, not an a-posteriori conversion.

Three practical consequences:

1. The "native" weights already weigh ~13GB: no quantization step, no extra loss. Community GGUF builds (unsloth, ggml-org) start from there, in sizes from 11.5 to 13.8GB depending on the chosen level.
2. Quantization does not degrade like a classic Q4: the model was post-trained already in MXFP4, so the benchmark numbers in the model card are the ones you get.
3. Inference engines support it natively: vLLM runs the original format without conversion; Ollama and llama.cpp use the GGUF builds.

How much it runs on 16GB of VRAM

The model file weighs between 11.5 and 13GB depending on the build. On a 16GB GPU (4060 Ti, 5060 Ti 16GB, 5070 Ti 16GB, RX 9070 XT) you have 3-4GB left for the KV cache: 8K to 32K of context depending on how far you push. Speed depends on memory bandwidth, as always: with 3.6B active parameters, even a 448 GB/s card handles decode without struggling.

No GPU? It works, but decode falls back to system RAM: expect around ten tokens per second on a modern dual-channel setup. Enough for interactive use, not for serving. On Apple Silicon, unified memory (16GB and up) is the natural setup: weights in memory, no swap.

Getting it running

With Ollama, two commands:

ollama pull gpt-oss:20b
ollama run gpt-oss:20b

Or in chat via LM Studio or Open WebUI pointing at the Ollama server. If you are serving multiple users, vLLM reads the original MXFP4 format straight from the Hugging Face repo.

The model supports "effort-adjustable" reasoning: in chat you can ask for more or less reasoning, and the model adapts the reasoning length to the task. It also works well with tool calling and structured outputs.

gpt-oss-20b vs Qwen3.8-27B vs Gemma 4 26B

If you have 24GB, the comparison that matters is this one:

	gpt-oss-20b	Qwen3.8-27B	Gemma 4 26B A4B
Architecture	MoE (32 experts, top-4)	Dense	MoE (128 experts, top-8)
Active parameters	3.6B	27.8B	3.8B
VRAM in Q4	~13GB	~16GB	~17GB
Native context	128K	262K	256K
Inputs	Text	Text + images + video	Text + images
License	Apache 2.0	Apache 2.0	Apache 2.0

The choices are different: Qwen3.8-27B is the "front-line" dense, with longer context and multimodality; Gemma 4 26B A4B is the MoE with the most generous context and 140+ languages; gpt-oss-20b is the one that fits in 16GB with the highest reasoning quality in the tier. If your goal is reasoning on 16GB hardware, it is the reference for now.

And the limits?

Honest take: the training data is "mostly English". Italian and French are understood and written fine, but quality in those languages stays behind models trained on heavier multilingual corpora, like Qwen or Gemma. If language is your main criterion, test both before choosing.

Another practical limit: 16GB means limited context in practice. The native 128K is theoretically possible, but with the weights taking 13GB the KV cache keeps you on 16-32K. If you need long contexts over big corpora, you need 24GB.

Finally, gpt-oss-120b (117B total, 5.1B active) runs on 80GB: H100, A100, or a 512GB Mac Studio M3 Ultra. A different world, but it exists in the same family.

Related articles

• Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB
• Quantization: get more from the same hardware
• Hardware for local AI: which GPU to choose

Sources

• Introducing gpt-oss — OpenAI
• gpt-oss-120b & gpt-oss-20b Model Card — arXiv
• openai/gpt-oss-20b — Hugging Face
• unsloth/gpt-oss-20b-GGUF — Hugging Face

In summary

gpt-oss-20b is OpenAI's first open reasoning model that fits in 16GB: 21B total with 3.6B active, native MXFP4 weights, up to 128K context and an Apache 2.0 license. Quantization is not a step you do: the model arrives in the 4-bit format it was trained in, so the model card numbers are the real ones. You get it running with Ollama in two commands. If your hardware is 16GB and you want reasoning, it is the model to try first; if language or multimodality is your criterion, Qwen3.8 and Gemma 4 remain serious alternatives.

Article generated with AI