ResearchPractical AIProjectsOpen SourceAbout
Research/An Eight-Hour Batch Window on One 16 GB Card: The Small Model Delivered 1.68 Times the Correct Answers per Hour, the Giant One Was More Accurate
Local AIMixture of ExpertsBatch ThroughputLatencyPre-RegisteredOriginal Research

An Eight-Hour Batch Window on One 16 GB Card: The Small Model Delivered 1.68 Times the Correct Answers per Hour, the Giant One Was More Accurate

I gave a 9B model and Niko1221's Strata Coder the same shuffled queue of 30,000 short business tasks (intent classification, invoice extraction, log triage) for eight hours each on one 16 GB graphics card, and counted correct items per hour. The 9B finished 24,805 items and the Coder 14,022. The Coder was 4.2 points more accurate, all of it on extraction, but the 9B delivered about 1.68 times the correct answers per hour. The models used different engines; a later same-Coder test measured a 6.8-fold engine gap.

Bart Porcincula · Published · Updated

Finding

Over a fixed 8 h window each, on an RTX 5060 Ti 16 GB at concurrency 1, Qwen3.5-9B Q4_K_M delivered 2,592 correct items per hour (83.6% accuracy) against 1,541 (87.9%) for Niko1221's Strata Coder IQ1_M. On the first 14,022 items both finished, Coder/9B correct per hour was 0.596 (95% CI 0.591 to 0.601, paired bootstrap, 10,000 draws), and the Coder's accuracy lead was +4.18 points (CI +3.69 to +4.69). By task, the ratio was 0.258 on intent, 0.906 on extraction and 0.540 on log triage; the accuracy gap came entirely from extraction (+11.7 points). Neither arm had a failed request.

Fixed 8-hour batch window, Qwen3.5-9B vs Niko1221's Strata Coder IQ1_M, 16 GB VRAM (Oct-Nov 2026)

One run per arm, one machine, one day each. RTX 5060 Ti 16 GB, 31.6 GB RAM. 9B: llama.cpp b11753, Q4_K_M, 32K context, thinking off. Coder: Strata Terminal 0.1.54, IQ1_M, 32K context, thinking off through a pass-through proxy. Same shuffled queue of 30,000 synthetic items (seed 0), temperature 0, concurrency 1, 300 s per-request cap, 8 h window per arm. Hypotheses were written before the first scored run.

MeasurementValueNote
Items finished in 8 h9B 24,805 vs Coder 14,022
Correct items per hour9B 2,592 vs Coder 1,541
Coder / 9B correct per hour (headline)0.59695% CI 0.591 to 0.601; the 9B delivers about 1.68 times as many
Accuracy9B 83.6% vs Coder 87.9%Coder +4.18 points, CI +3.69 to +4.69
Coder / 9B correct per hour, intent0.258accuracy tie: 69.0% vs 69.7%
Coder / 9B correct per hour, extraction0.906Coder +11.7 points accuracy (93.8% vs 82.1%)
Coder / 9B correct per hour, log triage0.540both 100% accurate
Median seconds per item9B 1.36 vs Coder 1.72p95 2.27 s vs 4.06 s
Peak video memory9B 6.9 GB vs Coder 15.5 GBCoder ran with 437 MiB of free RAM at its lowest, no failed requests

The short version

The last study in this bucket stopped a giant model because it was too slow to finish a coding benchmark, and suggested it might still suit work nobody waits on. This is that test. Both models got the same queue of short business tasks for a fixed eight hours, and the score was how many answers were correct per hour, not how good the best answer was.

  • The small model won on throughput. Qwen3.5-9B finished 24,805 items and got 20,736 right: 2,592 correct per hour. Niko1221's Strata Coder finished 14,022 and got 12,332 right: 1,541 per hour. The ratio, Coder over 9B, was 0.596 (95% interval 0.591 to 0.601), so the 9B delivered about 1.68 times as many correct answers.
  • The giant model was more accurate. 87.9% against 83.6%, a 4.2-point lead (interval +3.7 to +4.7).
  • That lead lives in one task. On invoice extraction the Coder was 11.7 points ahead. On intent classification it was a tie, and on log triage both scored 100%.
  • Nothing broke. Zero errors and zero timeouts in either window, even though the Coder machine fell to 437 MiB of free RAM.

Takeaway: for short, high-volume items, speed beat accuracy by a clear margin. The Coder only came close on extraction, where it was 0.906 of the 9B's throughput while winning on accuracy.

How it was run

Same card (RTX 5060 Ti, 16 GB), one run per arm, one day each, concurrency 1. The queue was 30,000 synthetic items in a fixed shuffled order: Banking77 intent labels, invoice field extraction, and log triage. Temperature 0, a 300-second cap per request, thinking off for both. The headline compared the first 14,022 items both arms finished, with a paired bootstrap (10,000 draws, seed 0). The five hypotheses, the pool and the software versions were all written down before the first scored item.

What the hypotheses said

HypothesisVerdict
The 9B has more correct items per hour overallSupported (0.596)
On log triage the Coder is at least 10 points more accurateRefuted (0 points; both perfect)
On intent the 9B beats the Coder by at least 1.5 times on correct per hourSupported (3.87 times)
The Coder's slowest 5% are at least 3 times its median, the 9B's under 2Refuted (Coder 2.36, 9B 1.66)
The gap narrows with a longer windowRefuted (ratio 0.59-0.60 at every hour mark)

The Coder was not the erratic model I expected. Its tail was longer than the 9B's, but both were steady across eight hours, and accuracy did not drift in either window.

Why the ratio moves with the task

The ratio is a property of the mix. On short intent items the Coder takes about 3.9 times as long per item (0.98 s against 0.25 s at the median) for no accuracy gain, so the 9B delivers roughly 3.9 times the correct answers. On extraction the Coder is only 1.3 times slower and 11.7 points more accurate, which nearly cancels. If your queue is mostly extraction, the answer here would be closer to even.

What I am not claiming

  • This is one run per arm, one machine, one day each. Engine versions are pinned, but background load was controlled only by an idle-GPU check before each window.
  • The tasks are synthetic, English-only, with short answers. It says nothing about long generation or reasoning-heavy work.
  • Intent accuracy of about 69% comes from exact-match scoring over a 77-label list. I did not analyse the misses or change the scoring.
  • One request at a time. Batched serving, which could change the speed picture for both models, was not measured.

Can a local AI process business paperwork overnight?

This test supports trying a reviewed overnight queue of short classification and extraction jobs. The smaller model completed more correct items per hour overall, but the larger model was more accurate on invoice extraction. The queue was synthetic and English-only; scanned documents, accounting integrations, and staff review time were not measured. Test your own document mix and count correction time before estimating a business saving.

Cite this research

Bart Porcincula. "An Eight-Hour Batch Window on One 16 GB Card: The Small Model Delivered 1.68 Times the Correct Answers per Hour, the Giant One Was More Accurate." Published 2026-11-01; updated 2026-11-08. https://porsync.com/research/eight-hour-batch-window-small-model-vs-giant-moe-correct-items-per-hour

Permanent article URL