Linear attention

Family of attention schemes (DeltaNet, Gated DeltaNet, Mamba, etc.) that replace the linearly growing KV cache with a fixed-size recurrent state, drastically reducing memory at long context. It's the technique behind hybrid-attention models like Qwen3.8-27B.