ResearchPractical AIProjectsOpen SourceAbout
Research/Which Small Coding AI Models Are Worth Running on a 16 GB Graphics Card?
Local AICoding ModelsBenchmarksAI Strategy

Which Small Coding AI Models Are Worth Running on a 16 GB Graphics Card?

I put eight coding models on one 16 GB graphics card and gave them ten small bug fixes through the same coding assistant. Six fixed every job. The clearest differences were in runtime and tool use. One model wrote out its tool calls as text, so the assistant could never run them. I used these results to pick candidates for the harder benchmark that followed.

Bart Porcincula · Published · Updated

Finding

On a 16 GB RTX 5060 Ti, six of eight models fixed all ten small Python jobs through OpenCode. Mellum 2 fixed seven and timed out on three. Qwen2.5-Coder 14B fixed zero after producing no executable tool calls. Three identical Bolt runs all passed, but wall time ranged from 234 to 338 seconds and tool errors from 2 to 24. The other timings are single runs.

Small coding models on a 16 GB graphics card, driven through OpenCode (Oct 2026)

Eight models (seven candidates plus a reference) served locally with llama.cpp b11753 on an RTX 5060 Ti with 16 GB of memory and a display attached, 32,768-token context. Each model ran ten small Python bug-fixing tasks through OpenCode 1.19.3 (600-second limit per task, a task counts as passed when the project's own tests pass and the test files are unmodified), plus single-shot code scores (HumanEval+ and MBPP+) and speed measurements. One model's loop was repeated three times as a noise check. Single runs otherwise.

MeasurementValueNote
Models that fixed all 10 tasks6 of 8Qwen3.5-9B (reference), gpt-oss-20b, Gemma 4 12B, Nemotron Bolt, Qwen3-Coder-30B, Devstral 24B
Mellum 2 (12B)7 of 10three tasks timed out; it generated about 4 tokens a second on this setup
Qwen2.5-Coder 14B0 of 10wrote its tool calls as plain text, so none were ever executed, despite a strong single-shot score (0.872)
Fastest on the tasksgpt-oss-20b, 156 s totalthe 9B reference took 176 s and Gemma 4 12B 219 s: indistinguishable given the noise
Slowest loop-capable modelDevstral 24B, 1,896 s totaldecode fell from 10.2 to 3.9 tokens a second as the context filled to 28K, with 15.7 GB of the 16 GB resident
Decode speed with 28K tokens of context104 / 76 / 47 / 4 tok/sgpt-oss-20b / Qwen3-Coder-30B / Gemma 4 12B / Devstral 24B
Single-shot HumanEval+ range0.665 to 0.890Gemma 4 12B best (0.890), gpt-oss-20b lowest (0.665); yet gpt-oss passed all 10 loop tasks
Noise check (one model, same settings, 3 runs)wall 234 to 338 s; tool errors 2 to 24pass rate did not move (10/10 each time), so timing gaps under about 45% are ties
Qwen3-Coder-30B's odd task390 tool calls on one taskthe fix landed, then it kept calling tools until the 600 s limit; counted as a pass, 9 of 10 if counted strictly
Models proposed for the next study (sprocket)Gemma 4 12B and gpt-oss-20bfast, clean tool use, 8 GB and 3.7 GB of memory headroom; a proposal, pending my sign-off

The short version

If you want an AI coding assistant that runs on your own computer, with no code leaving the machine and no per-use bill, the first question is which model. Online guides disagree, and most were not tested on a real consumer card. So I picked a common one, an RTX 5060 Ti with 16 GB of memory, and tested.

I took seven candidate coding models plus a reference, put each inside the same coding assistant (OpenCode), and gave each ten small bug-fixing jobs on little Python projects. A job counts as done only when the project's own tests pass afterwards and the test files are untouched.

The headline: the jobs did not separate the models. Six of the eight fixed all ten. What separated them was speed, how cleanly they used their tools, and one model that could not use tools at all.

What happened

ModelJobs fixedTotal timeNotes
Qwen3.5-9B (reference)10 of 10176 sno tool errors
gpt-oss-20b10 of 10156 sfastest
Gemma 4 12B10 of 10219 s1 tool error
Nemotron Bolt (community-shrunk to fit)10 of 10338 s24 tool errors in this run, see the noise check
Qwen3-Coder-30B10 of 10784 sone task ran to the time limit (390 tool calls)
Devstral 24B10 of 101,896 svery slow once the context fills
Mellum 2 12B7 of 103,198 sabout 4 tokens a second, three jobs timed out
Qwen2.5-Coder 14B0 of 10164 snever produced a tool call that could run

Fixing the code is not the same as using the tools. Qwen2.5-Coder 14B scores well when it answers a coding question in one go (0.872 on a standard test, third best here). Inside the assistant it scored zero, because it wrote its tool calls out as ordinary text instead of in the form the assistant executes. A model that is good at code but cannot drive the tools is no use as an assistant.

Speed is where the size shows. The two biggest files on this card, Devstral 24B and Qwen3-Coder-30B, sit close to the 16 GB limit (15.1 and 15.7 GB in use). Devstral, a dense model, slowed from 10 to under 4 tokens a second as the conversation grew, about ten times slower than the rest. Qwen3-Coder, which only activates a small part of itself for each word, stayed fast (76 to 112 tokens a second). Being near the ceiling is not enough on its own to slow a model down.

Strong on one test does not mean strong on the other. gpt-oss-20b had the lowest single-shot coding score here (0.665) and was the fastest, cleanest model on the bug-fixing jobs. The two kinds of test measure different things.

Where to get these exact models

Every model above is linked to the Hugging Face repository I downloaded it from, so you can read each model card and licence yourself. The quantisation I ran is the one that fit 16 GB with room for context: Q3_K_M for Devstral 24B, UD-Q2_K_XL for Qwen3-Coder-30B, MXFP4 for gpt-oss-20b, Q4_K_M for Gemma 4 12B, Mellum 2, Qwen2.5-Coder 14B and the Qwen3.5-9B reference, and IQ2_XXS for Nemotron Bolt. I tested how that community build fits and what it scores in the separate Bolt write-up.

The noise check

I ran one model (Bolt) through the ten jobs three times with identical settings. It fixed all ten every time. But total time ranged from 234 to 338 seconds (up to 45% apart) and tool errors from 2 to 24. So a single run's timing or error count is not a ranking. By that yardstick, the 9B reference, gpt-oss and Gemma are indistinguishable on these jobs, and only Devstral and Mellum are clearly slower.

What I would pick

For a 16 GB card, Gemma 4 12B and gpt-oss-20b: fast, clean with tools, and with several gigabytes of memory to spare for longer conversations. They are also different designs from different makers, which makes them a more informative pair to test. Qwen3-Coder-30B is a stronger option if you watch for runaway tool loops, and Nemotron Bolt if you need long context. I would not use Devstral 24B, Mellum 2 or Qwen2.5-Coder 14B on this card in this setup.

What this does not show

  • The jobs are easy and mine. Six of eight models got a perfect score, so this ranks speed and tidiness, not coding skill. It is a selection table, not a leaderboard.
  • Most numbers are one run. Only Bolt's loop was repeated, and nothing here has confidence intervals.
  • Context was tested up to about 28K tokens, not each model's maximum.
  • A few things were not diagnosed: why Mellum 2 was so slow on this software build, and why Qwen2.5-Coder wrote its tool calls as text (the template, the server or the model).
  • Disclosed adjustments: the standard code-scoring tool needed three Windows patches (it dropped its request, memory and per-test time limits), and the 9B reference ran from a slightly different file than my everyday copy, because that copy does not load on this software build. Mellum 2's second standard test was cut short, so it has no score for it.

What I tested next

I put the surviving models through a harder ladder of public benchmarks, with gates that stop a model before the next rung. I also repeated the original jobs and added harder ones. On the original ten, gpt-oss-20b dropped two jobs across seven runs. Its total times overlapped the 9B's, so the 20-second lead in the table was too small to call.

Cite this research

Bart Porcincula. "Which Small Coding AI Models Are Worth Running on a 16 GB Graphics Card?." Published 2026-10-25; updated 2026-11-07. https://porsync.com/research/which-small-coding-ai-models-run-on-a-16gb-card

Permanent article URL