A 74B Coding Model on a 16 GB Card: It Fit With Room for a 116K-Token Prompt
I ran a community build of Nemotron Bolt on a 16 GB graphics card. It used 14.8 GB of video memory with 116,100 tokens in the prompt and generated about 50 tokens a second. Its coding and tool-call scores came within 2.5 points of the builder's reported scores. On 24 everyday questions, my smaller 9B model got five more right.
Bart Porcincula · Published
Finding
Nemotron Bolt REAP50 IQ2_XXS ran on an RTX 5060 Ti 16 GB with stock llama.cpp b11753. At a 128K context setting with 116.1K tokens actually in the prompt, it used 14.8 GB of video memory and decoded at 50.3 tokens per second. It scored 86.0% on HumanEval+, 69.6% on MBPP+ and 90.2% on BFCL simple call correctness. It answered 19 of 24 non-code questions correctly versus 24 of 24 for Qwen3.5-9B.
Nemotron Bolt fit, speed and benchmark reproduction on 16 GB (Oct 2026)
Single-slot uncached prompt-length sweep, three published benchmark comparisons and small non-code and tool-loop probes. Coding and tool-call benchmarks used four slots with 32K tokens per slot, with reasoning disabled.
| Measurement | Value | Note |
|---|---|---|
| Largest prompt tested | 116.1K tokens | 128K context setting, 14.8 GB video memory |
| Decode at 116.1K | 50.3 tok/s | |
| HumanEval+ / MBPP+ | 86.0% / 69.6% | builder reported 83.5% / 68.8% |
| BFCL simple call correctness | 90.2% | builder reported 90.8% |
| Everyday questions | 19/24 vs 24/24 | Bolt vs Qwen3.5-9B |
The short version
I wanted to know how much room a community build of Nemotron Bolt would leave on an ordinary 16 GB card. The builder shrank the original model by removing some of its experts, the parts of a model that get called on for different inputs, and storing the remaining weights at low precision. I used the REAP50 IQ2_XXS build from the community model card and ran it on stock llama.cpp with a display attached.
At a 128K context setting, a 116,100-token prompt brought total video memory use to 14.8 GB. The model generated 50.3 tokens a second there, down from 57.4 with a 17,700-token prompt. I saw no sudden slowdown from running out of video memory across the lengths I tested. The long prompt was filler, so this measures how much text the model can load and process. How well it uses that much text is a separate question.
How well did the scores travel?
| Test | My 16 GB card | Builder's DGX Spark |
|---|---|---|
| HumanEval+ (164 coding problems) | 86.0% | 83.5% |
| MBPP+ (378 coding problems) | 69.6% | 68.8% |
| BFCL simple (400 tool calls) | 90.2% correct | 90.8% correct |
These are single runs with different software builds. I made three compatibility patches to run EvalPlus on Windows, which removed its request timeout, memory cap and per-test timeout. My BFCL checker also differed from the upstream checker by about one point in an offline check. The scores are close enough for the practical question I had, which was whether the published results would carry over to my machine.
Everyday questions were a different test
Bolt got 19 of 24 right on a small set of arithmetic, facts, translation and text transformations. Qwen3.5-9B got all 24. Bolt also answered all six supported requests in my little tool loop, but both models struggled to return valid final JSON on the harder requests under a four-call cap. The probes are too small to rank the models broadly. I kept Bolt in the OpenCode comparison to see how it handled actual edits.
Cite this research
Bart Porcincula. "A 74B Coding Model on a 16 GB Card: It Fit With Room for a 116K-Token Prompt." Published 2026-11-07; updated 2026-11-07. https://porsync.com/research/nemotron-bolt-74b-on-a-16gb-graphics-card
Permanent article URL