Red Banana Lab · Field notes No. 7

The 32B Sweet Spot: Precision, Throughput, and Context on AMD Strix Halo

Red Banana Lab, New Delhi · 14 October 2026 · production verification on our 128 GB workstation, Radeon 8060S Vulkan backend

Field notes No. 1 and No. 2 documented our tests on frontier-scale Mixture-of-Experts weights (80B to 235B) running on unified memory. Those models deliver remarkable reasoning, but their weight files occupy 60 to 98 GB of storage and require deliberate thermal discipline. This note tests the opposite end of the spectrum: can a modern 32B dense model serve as a permanent, zero-compromise daily driver on the same 128 GB workstation (AMD Ryzen AI Max+ 395, 128 GB unified memory, Radeon 8060S graphics)?

The Contenders

Model	Quant	Weight File	Architecture Notes
qwen2.5:32b (Instruct)	Q4_K_M	19.8 GB	Dense 32.8B, 32k context, general instruction tuning
qwen2.5-coder:32b	Q4_K_M	19.8 GB	Dense 32.8B, 32k context, code and syntax specialization
gpt-oss-120b (Baseline)	MXFP4	61.0 GB	MoE, 5.1B active, separate reasoning channel

Headline Numbers

Metric	Qwen2.5 32B Instruct	Qwen2.5 Coder 32B	gpt-oss-120b
Generation throughput, tok/s	7.90	8.12	51.0
Prompt ingest rate, tok/s	90.9	92.4	644.0
Time to first token (short prompt)	0.61 s	0.64 s	0.80 s
Resident VRAM footprint (32k ctx)	28.0 GB	28.0 GB	70.0 GB
VRAM headroom remaining	88.0 GB	88.0 GB	46.0 GB
Quality gauntlet (planted answers)	5/5	5/5	4/5*

Finding 1: The 28 GB Resident Ceiling

A 32k context window fits with 88 GB to spare At Q4_K_M, the 32B model weights consume 19.8 GB. Allocating the full 32,768-token KV cache buffer directly inside the Radeon 8060S Vulkan window takes an additional 8.19 GB. The total resident footprint is exactly 28.0 GB. On a 116 GB addressable VRAM pool, this leaves 88 GB completely unoccupied. The system can keep this model resident indefinitely without paging, memory compression, or swap pressure.

Finding 2: The End of Empty Reasoning Channels

In Field Notes No. 1, gpt-oss-120b failed a critical task because its completion budget was consumed entirely inside its private reasoning channel, leaving the content channel blank. In contrast, Qwen2.5-32B emits directly to the response channel. Across our entire test battery (multi-digit integer summation, bracket validation functions, and structured JSON invoice parsing), completion was immediate and zero empty responses occurred.

Finding 3: Dense Bandwidth vs Sparse Activation

The speed gap between 7.9 tok/s (32B dense) and 51 tok/s (120B MoE) illustrates the physical limits of memory bandwidth. Because 32B is a dense model, every single token generation pass reads all ~20 GB of quantized weights across the LPDDR5X bus (~256 GB/s bandwidth). An MoE model with 5.1B active parameters streams only ~4 GB per token, generating six times faster despite having four times the parameter volume.

Finding 4: Perfect Arithmetic and Execution Pass

Quantization at 4-bit frequently degrades arithmetic consistency. In our planted verification run, Qwen2.5-32B solved the multi-term addition 48291 + 19482 + 91023 with zero hesitation, returning the exact integer 158796 without extraneous conversational boilerplate. Its generated Python balancing function passed all test fixtures on the first execution.

What We Deployed

We have integrated Qwen2.5-32B as our permanent resident service on port 11434. It handles everyday agent routing, document parsing, and script generation. When complex multi-file architectural reviews or heavy research synthesis are requested, we route upward to 120B and 122B sparse models on port 8080. This gives us instant responsiveness for ordinary work while preserving 88 GB of VRAM headroom for the heavy weights.