ResearchPractical AIProjectsOpen SourceAbout
Research/A 120B Model Fit on My 16 GB Card. Reliable Coding Was Still Unresolved
Local AICoding AgentsEvaluationOriginal Research

A 120B Model Fit on My 16 GB Card. Reliable Coding Was Still Unresolved

Edge0 ran interactively. A separate routing-accounting retry passed 15/15 final checks, with two frontier repairs, but showed no speed win. The router remains frontier-only.

Bart Porcincula · Published · Updated

Finding

Edge0-120B-q3k passed 27/50 and 21/50 harder synthetic coding attempts in separate live-context runs without execution errors; earlier runs encountered OOM responses and prefill timeouts. Separate CLI accounting passed 21/21 direct, 36/36 shadow and 15/15 retry final checks; two retry tasks needed frontier repair. The retry showed no speed win or promotion conclusion. Broader comparisons are outside this selected release.

A 120B Model Fit on My 16 GB Card. Reliable Coding Was Still Unresolved (November 2026)

Interim Edge0 coding-agent evaluation on RTX 5060 Ti 16 GB and 32 GB RAM. Ten synthetic fixtures, five attempts each per run; runs are separate observations, not a controlled retry ablation.

MeasurementValueNote
Later harder-task runs27/50; 21/50separate runs with identical settings; zero execution errors
Real-fix replay0/12 cumulative; 7/12 live-contextfour historical fixes, three attempts each; exploratory
Thinking-off HumanEval+ / MBPP+0.884 / 0.669one greedy evaluation
Router defaultfrontier-onlyinitial accounting measured; broader quota and host/shadow evidence remain open

A model can fit and still need supervision

A 120-billion-parameter model answered on my desktop: one RTX 5060 Ti with 16 GB of graphics memory and 32 GB of system RAM. That was the first result. The more useful question was whether it could finish coding work reliably.

On ten harder maintenance tasks, two later runs with identical settings passed 27 of 50 attempts and 21 of 50. Neither run recorded an execution error. The unsuccessful attempts mostly repeated tool calls until the agent stopped them. Fast text generation and a successful load did not establish a dependable coding assistant.

This is an interim report. The selected public update adds CLI accounting and a separate local retry. It does not resolve the evidence needed to promote local routing; other frontier follow-ups are outside this release.

What fit on the machine

The model was Edge0-120B-q3k, served through Edge0's inference stack. Initial thinking-enabled probes generated about 45 tokens per second, with longer-prompt processing around 1,000 tokens per second. Tokens are pieces of text; reasoning text counts too, so that speed does not mean 45 tokens of useful final answer each second.

Memory was tight. The initial loads used about 15,640–15,655 MiB of graphics memory, and available system RAM briefly approached zero. Warm-cache loading took roughly 66–100 seconds at the tested context settings. A cold boot was not timed. The downloaded model directory occupied about 86 GB, including an additional required weights file.

The short probes completed without faults. Later coding runs encountered out-of-memory errors and requests that stalled until the client timed out. “It loads” was therefore a narrower claim than “it is stable under agent work.”

Short coding answers were stronger than sustained repair

With thinking disabled, the model scored 0.884 on HumanEval+ and 0.669 on MBPP+, two suites that check generated code against tests. Those figures did not make the largest model a clear winner over the smaller models already tested on this card. A separate thinking-enabled HumanEval+ run scored 0.616 and took about twelve times the generation time. A later scoring pass on the 106 archived thinking-enabled MBPP+ samples returned 87 passes, versus 84 with thinking off on the same IDs. That interrupted, nonrandom subset is not a complete MBPP+ result or a reliable thinking benefit estimate.

The agent experiment asked for more: inspect files, make changes, run tests and finish a repair. Its original seven small fixtures gave Edge0 success rates of 67–96% per fixture. Sonnet and Haiku through the Claude CLI each passed 70/70 on those original fixtures. Their orchestration paths differed, and the fixtures were small. Those results cannot supply the missing comparison on harder work.

A budget mistake changed what the agent could attempt

An agent sends context back to the model each turn. The original harness added those repeated prompt tokens into a cumulative budget. A long interaction could hit its limit even when the current prompt still fit comfortably.

The revised harness limits the live prompt instead. That changed the stopping rule, so I report each run separately rather than presenting the sequence as one controlled experiment.

RunBudget and clientAttempts passing tests
OriginalCumulative, 24,000 tokens18/50
Larger budgetCumulative, 30,000 tokens7/50
First live-context runCurrent prompt capped12/50
Later runLive-context, retry-capable client27/50
RepeatSame settings as preceding run21/50

The first live-context run had eight execution errors: two out-of-memory responses and six prefill stalls reaching the 180-second timeout. Low available host memory was a plausible contributor, but this was not an experiment isolating memory pressure.

The client gained a single retry for timeouts and explicit out-of-memory responses. However, the two later runs had no logged qualifying faults. Their improvement cannot be credited to the retry. A subsequent controlled proxy test recovered from one injected OOM response and one injected timeout, then completed real Edge0 inference. It also correctly surfaced generic errors and exhausted retries. Recovery from natural memory exhaustion or a real prefill stall remains unvalidated.

In the two later runs, every unsuccessful attempt ended in repeated-tool-call detection except one turn-limit stop per run. Three individual tasks swung by two or three passes out of five. Ten repeated fixtures provide much less breadth than a hundred different maintenance requests.

Real fixes were encouraging, but few

Four historical fixes qualified by failing their tests before the reference fix and passing afterward. Three attempts per task produced 0/12 passes under cumulative budgeting and 7/12 under live-context budgeting.

That is useful evidence that the harness budget matters. It is not a precise causal estimate: the replay was exploratory, small, and drawn from one author's repositories. Frontier follow-ups on these fixes are outside the selected public release.

What remains before local routing is justified

The router still defaults to frontier. A category label inferred from an easy fixture cannot establish that a harder request will succeed locally. A successful implementation and offline tests also cannot establish a reliable live handoff.

The initial plugin-path accounting is now measured on the original small tasks, as described below. Broader quota impact, sufficient host/shadow evidence on local-table decisions and held-out routing validation remain open. Other harder-task frontier follow-ups are outside this selected release.

A six-minute local follow-up found just 0.011–0.347 GiB of available RAM. All seven small inference probes succeeded, but the unchanged 2 GiB health floor blocked all three routing checks. This validates the gate behavior, not a safe threshold or completed coding task. One simple marker-retrieval request at 28,036 input tokens also succeeded in 25.3 seconds. Health calibration and natural-fault recovery remain open. Cold-cache loading, an enforced RAM cap and other model configurations were not tested either. These are separate from the quota-blocked frontier calls.

For now, the result supports a constrained conclusion: this large model can run interactively on the machine, while repeated failures and memory pressure prevent treating that fact as evidence of unattended coding reliability. The expert-cache follow-up asks a separate speed question using a different model and backend.

What the CLI accounting update measured

These are separate cohorts of small synthetic tasks, not a new harder-task reliability benchmark. B0 used direct Sonnet subscription CLI calls. B1 added a shadow router that recorded decisions without delegating. The original B2 smoke and the later B2 retry remain separate.

CohortFinal independent passesLocal execution
Direct CLI baseline B021/21None
Shadow B136/36None
Original B2 smoke1/1None; remaining 14 skipped
Separate B2 retry15/1513 local successes; two frontier repairs

B1's 36 decision/outcome pairs agreed with the runner's pass results, but its self-reported timing was unreliable. Relative to B0, the fixture-balanced mean of per-fixture median differences was +99,410.5 cache-read tokens, -4,040.9 cache-write tokens, +547.8 output tokens and +13.094 seconds. This is a descriptive comparison, not a paired causal estimate or a measurement of subscription quota consumption.

The cold-start label was misleading: recent_http400 came from a synthetic in-memory cooldown initialized when the process started, not a real HTTP 400. The successful retry waited through the unchanged five-minute cooldown in the same MCP process. RAM floor 0 was an authorized accounting deviation, not a recommended operating threshold or controlled calibration. Available RAM was only 0.017-0.894 GiB in warmup samples; all 106 logged HTTP responses were 200.

The retry completed 15 final independent checks. Thirteen local proposals passed; two failed locally and were repaired by frontier. One earlier quota-interrupted local attempt is archived and excluded, including after its key was rerun. That makes 16 local attempts across the archive and completed cohort, not 16 completed trials.

The local-ledger median was 19.703 seconds, while median CLI end-to-end time was 336.125 seconds including startup waits. Compared with B1, fixture-balanced mean median differences were -67,376.1 cache-read, -1,186.8 cache-write and -129.3 output tokens, but +306.625 seconds overall and +6.113 seconds after subtracting each trial's observed waits. There is no speed win and no promotion conclusion.

Fixed CLI context is a major caveat: roughly 25k cache-write and 92-135k cache-read tokens surrounded only 250-500 task-output tokens. Unweighted token arithmetic does not establish subscription quota savings, billed spend or the effect of routing on larger tasks.

E39 items 1 and 2 remain open: broader plugin-path accounting and actual quota impact, and sufficient host/shadow integration evidence on local-table decisions. Health calibration, natural-fault recovery and held-out routing evidence also remain open. The default stays frontier-only.

Evidence

The Edge0 coding-agent results record contains hardware pins, benchmark results and the limits of the verification. The router record tracks implementation and missing measurements. The public package includes the local-validation observations and partial benchmark summary. Raw benchmark solutions, private implementation code and agent transcripts are excluded; later raw agent logs remain on the run machine. The release notes state what can be independently checked. The reliability follow-up remains open.

The new B0 baseline, B1 and original B2 accounting, and separate B2 retry publish curated usage/outcome rows, configuration and offline analysis. Original-source hashes and transformations are recorded in the manifest. Fixtures, reference solutions, private implementation and raw traces are withheld. Readers can recompute aggregates, but cannot independently rescore generated code from this release.

Cite this research

Bart Porcincula. "A 120B Model Fit on My 16 GB Card. Reliable Coding Was Still Unresolved." Published 2026-11-15; updated 2026-11-19. https://porsync.com/research/edge0-120b-16gb-fits-but-agent-reliability-unresolved

Permanent article URL