Chinese Frontier Models in Production: Benchmarking Kimi K2.5, Qwen3-Coder-480B, GLM-5, DeepSeek V3.2, and MiniMax Against Western Models on Live Agency Workloads
Social media is full of promotional reels claiming a single Chinese lab just made Western frontier models obsolete. The reel that started this trial says: "A Chinese company just dropped a FREE open-source AI model that's beating Claude and GPT at coding." Rather than trusting reels or synthetic leaderboards, we ran five Chinese frontier model families head to head against two Western baselines through our production router on three tasks from our own agency queue.
The three live agency tasks
| Task ID | Task, as named in the log | Pass rule |
|---|---|---|
T1 | T1_starbasedb_ts_bug: a TypeScript bug fix on an open-source SQLite project | the passed flag in the log |
T2 | T2_threejs_patina_shader: a Three.js patina shader | the passed flag in the log |
T3 | T3_seo_audit_json: an SEO audit returned as JSON | the passed flag in the log |
The log stores, per model and task, the return code, latency, output characters, estimated output tokens, estimated tokens per second and a passed flag. It does not store the prompts, so the tasks are described here only by their log names.
Headline results (first run, then the follow-up run)
| Model | Origin | Task 1 | Task 2 | Task 3 | Total time | Total out tokens | Pass rate |
|---|---|---|---|---|---|---|---|
Qwen3-Coder-480B-A35B | CN (Alibaba) | 1.71 s (64 tok) | 2.02 s (52 tok) | 1.73 s (22 tok) | 5.46 s | 138 | 3/3 |
Kimi K2.5 (moonshotai.kimi-k2.5) | CN (Moonshot) | 2.76 s (81 tok) | 4.67 s (81 tok) | 2.83 s (29 tok) | 10.26 s | 191 | 3/3 |
GLM-5 (zai.glm-5) | CN (Zhipu AI) | 4.77 s (264 tok) | 3.06 s (60 tok) | 2.27 s (35 tok) | 10.10 s | 359 | 3/3 |
DeepSeek V3.2 (first run) | CN (DeepSeek) | 11.51 s (FAIL, rc 2, 205 tok) | 4.55 s (56 tok) | 3.29 s (27 tok) | 19.35 s | 288 | 2/3 |
DeepSeek V3.2 (re-run at 1,500 max_tokens) | CN (DeepSeek) | 6.30 s (186 tok) | 4.39 s (63 tok) | 15.65 s (21 tok) | 26.34 s | 270 | 3/3 |
MiniMax M2.5 (minimax.minimax-m2.5, re-run at 1,024 max_tokens) | CN (MiniMax) | 8.97 s (0 visible) | 8.40 s (0 visible) | 12.38 s (0 visible) | 29.75 s | 0 visible | 0/3 |
MiniMax-M3 (SambaNova, first run and 2,048 re-run) | CN (MiniMax) | 1.0 s (rc 1, 0 chars) | 0.6 s (rc 1, 0 chars) | 0.56 s (rc 1, 0 chars) | 2.16 s | 0 | 0/3 |
Claude Sonnet 4.6 (Western baseline) | US (Anthropic) | 2.43 s (98 tok) | 2.71 s (75 tok) | 2.92 s (41 tok) | 8.06 s | 214 | 3/3 |
GPT-OSS-120B (Groq LPU baseline) | US (OpenAI OSS) | 1.82 s (222 tok) | 1.62 s (94 tok) | 1.46 s (32 tok) | 4.90 s | 348 | 3/3 |
Totals are the sum of the three task rows. Token counts are the log's estimates. The first-run token budget is not recorded in the log; the re-runs record 1,500 (DeepSeek), 1,024 (MiniMax M2.5) and 2,048 (MiniMax-M3).
Qwen3-Coder-480B-A35B (480B total, 35B active, per our router notes) was the standout engineering model in the Chinese cohort. It passed all three tasks in 5.46 s cumulative wall-clock time against 8.06 s for Claude Sonnet 4.6, 32.3% less, while emitting 138 estimated output tokens across the three tasks against 214, 35.5% fewer.moonshotai.kimi-k2.5 passed 3 of 3 tasks with 191 total output tokens, leaner than Claude Sonnet 4.6's 214. zai.glm-5 also passed 3 of 3 and recorded the highest estimated generation speed among the Bedrock models in the log, 55.3 tokens per second on Task 1. For comparison, GPT-OSS-120B on Groq's LPU recorded 122.0 tokens per second on the same task.Field Notes No. 1 flagged the empty-answer channel risk on a local reasoning model. Our cloud trial met the cloud equivalent on minimax.minimax-m2.5, moonshotai.kimi-k2.5 and zai.glm-5 on AWS Bedrock's Converse API:
- Token starvation under small caps: with a small
--max-tokensbudget (such as 60 on a smoke check), these models spend about 150 to 250 tokens inside a hiddenreasoningContentblock before emitting a visible character, so the visibletextcomes back empty. - The
content[0]indexing trap: Bedrock Converse places thereasoningContentobject atoutput.message.content[0]and the visible answer after it. A caller that reads onlycontent[0].textsees an empty string even though the HTTP call succeeded (rc=0). - The fix in our router: a minimum budget
eff_max_tokens = max(max_tokens, 1024)for those three models, and extraction of everytextblock inoutput.message.content[], falling back to the reasoning text if no visible block exists.
What the fix did and did not do, from the logs: Kimi K2.5 and GLM-5 passed 3 of 3 in the main run. DeepSeek V3.2 failed Task 1 in the first run (rc 2 after 11.51 s) and passed 3 of 3 when re-run at 1,500 max_tokens. MiniMax M2.5 at 1,024 max_tokens still returned 0 visible characters on all three tasks in the follow-up run, with rc 0, and MiniMax-M3 on SambaNova failed all three with rc 1 at both budgets. So the floor is not a universal fix: five of the six families pass, MiniMax does not yet.
What we deployed in our router
All six Chinese model families are wired as first-class flags in our frontier router on both our first and second workstations, served through AWS Bedrock (ap-southeast-2) and Groq's free LPU tier. Each flag answered an identity probe and a 17 x 23 = 391 check live on 19 October 2026.
| Router flag | Primary and fallback models | Note from this trial |
|---|---|---|
--qwen-coder | qwen.qwen3-coder-480b-a35b-v1:0 + qwen.qwen3-coder-next | fastest 3 of 3 pass in the trial (5.46 s) |
--kimi | moonshotai.kimi-k2.5 | 3 of 3, 191 output tokens; 1,024-token floor applied |
--glm | zai.glm-5 | 3 of 3; 55.3 tokens per second on Task 1; 1,024-token floor applied |
--deepseek | deepseek.v3.2 | 2 of 3 in the first run, 3 of 3 at 1,500 max_tokens |
--minimax | minimax.minimax-m2.5 + SambaNova MiniMax-M3 | reasoningContent plus text extraction; 0 of 3 visible in this trial |
--qwen | qwen.qwen3-235b-a22b-2507-v1:0 + Groq qwen/qwen3.8-27b | identity probe only; not in the three-task run |
Sources
All figures above are copied from our own lab records of 19 October 2026: the first benchmark run (all seven models), the follow-up run (the DeepSeek and MiniMax re-runs), and the Chinese models and frontier router report (flags, fallbacks, the reasoningContent fix, identity probes, the reel quote). Corrections welcome.