$

llm-rate

Thursday, 8 October 2026

In the last 24 hours we dispatched 1,556 tasks across 4 models. Here's what we picked, and why.

What we ran

An autonomous AI fleet, written in TypeScript, picks a model per task using a complexity router. No vibes, no PR team. This is the actual production output of that router:

ModelDispatchesShareWhy this one
01 claude-sonnet-4-6 1,066 68.5% implementation (standard)
02 claude-haiku-4-5 278 17.9% implementation (light)
03 gpt-5.4-mini 189 12.1% implementation (codex pool)
04 claude-opus-4-6 23 1.5% implementation (high complexity)

Window: 24h to 2026-05-16T00:00:00Z. Source: daemon routing logs. The router writes a decision per dispatch; we parsed 1556 of them.

If you don't have a router, here are the picks per common task

Filtered from arena.ai's leaderboard plus published API prices. Filter thresholds are listed under each tab; arguable. Treat this as a starting shortlist, not a verdict.

Fast, volume conversations. Latency-sensitive. Margin matters.

Best value

gemma-3n-e4b-it

Google · quality 1305.5 · $0.10/M blended

Best quality

gemini-3.8-flash-high

Google · quality 1499.0 · $2.85/M blended

Filter: Filtered to blended price ≤ $5 per million tokens and quality ≥ 1300 Arena. Ranked by value: most quality per dollar wins. 154 models survived.

Model Quality Ctx In /1M Out /1M Value ↓
01 gemma-3n-e4b-itvalueGoogle 1305.5 33k $0.06 $0.12 299538.5
02 glm-5.3-flashZ.ai 1471.1 1.0M $0.06 $0.20 298144.0
03 gemma-3-12b-itGoogle 1334.2 131k $0.05 $0.15 278503.7
04 gpt-5-nano-highOpenAI 1319.7 400k $0.03 $0.20 216732.3
05 mimo-v2.6-flashXiaomi 1455.2 1.0M $0.14 $0.28 191260.6
06 nvidia-nemotron-3-nano-30b-a3b-bf16Nvidia 1347.7 262k $0.06 $0.24 186941.2
07 mimo-v2.5Xiaomi 1427.6 1.1M $0.14 $0.28 179653.0
08 mimo-v2-flash (non-thinking)Xiaomi 1411.0 262k $0.10 $0.30 171230.6
09 step-3.5-flashStepFun 1402.9 262k $0.10 $0.30 167862.7
10 granite-4.2-8bIBM 1319.6 131k $0.06 $0.25 165579.5
11 mimo-v2-flash (thinking)Xiaomi 1394.7 262k $0.10 $0.30 164471.1
12 qwen3-30b-a3b-instruct-2507Alibaba 1383.8 262k $0.10 $0.30 159913.6
13 gemini-3.8-flash-highqualityGoogle 1499.0 1.0M $0.75 $3.75 17507.2
14 mimo-v2.6-proXiaomi 1491.7 1.1M $0.43 $0.87 66485.3
15 muse-spark-1.3-maxMeta 1489.6 1.0M $1.25 $4.25 14614.0

What this is, and isn't

Right now this is filter-on-arena.ai plus a public log of what we ran. Arena Elo measures pairwise human preference on short prompts. It does not measure: whether a model produces valid JSON under a schema, whether it hallucinates function names, whether it refuses queries it shouldn't, latency p99, rate-limit behaviour. Production teams need those signals.

We're building a benchmark runner — fixed prompt suites for RAG, structured extraction, code refactoring, function calling — run daily against every model. Raw inputs, outputs, judge rationale, costs published. When that lands, the "picks" section gets its real backing. Until then, the picks section is opinion with a citation, not measurement.