ResearchPractical AIProjectsOpen SourceAbout
Research/Paste the Whole Folder or Retrieve the Relevant Notes? On a Small Model, Pasting Won Up to 120 Notes
EvaluationRetrievalRAGContext WindowLocal AIPre-RegisteredOriginal Research

Paste the Whole Folder or Retrieve the Relevant Notes? On a Small Model, Pasting Won Up to 120 Notes

Part 4 of 5. The standard advice for working over a pile of notes is retrieval: find the few relevant ones and show only those. I tested that against simply pasting the whole folder, from 10 to 120 notes (about 46,000 tokens). Pasting was more accurate at every size. Retrieval tied it on single-note questions and was 2 to 20 times faster. The entire gap was questions that need two notes.

Bart Porcincula · Published · Updated

Finding

On 60 questions per size over a synthetic project-notes folder (10, 30, 60 and 120 notes), Qwen3.5-9B with the whole folder pasted scored 97%, 95%, 83% and 77%, against 67% for the best retrieval arm at every size from 30 up (70-77% at 10 notes), with retrieval at 100% on single-note and distractor questions and 0% on multi-hop questions except at 10 notes, where top-8 scored 10% and hybrid scored 30%; pasting was 2.0 to 20.1 times slower per question (1.5 s to 18.4 s against 0.7 to 0.9 s).

Whole-folder prompt vs top-k retrieval, 10 to 120 notes (Oct 2026)

A generated folder of project notes (10, 30, 60 or 120 notes: one fifth as many vendors, five notes each) and 60 questions per size: 20 single-note, 20 with a near-duplicate distractor note, and 20 multi-hop needing two notes. Arms: whole folder pasted (ST), top-3 or top-8 by embedding similarity (R3, R8), top-3 from a rank fusion of keyword search and embeddings (R3+BM), and an oracle given only the right notes (ORA). Exact-match scoring. Qwen3.5-9B Q4_K_M, f16 cache, temperature 0, thinking off, prompt cache off for every arm; embeddings from nomic-embed-text. Paired bootstrap, 10,000 draws.

MeasurementValueNote
Whole folder pasted, accuracy at 10 / 30 / 60 / 120 notes97% / 95% / 83% / 77%
Best retrieval arm, accuracy at 10 / 30 / 60 / 120 notes77% / 67% / 67% / 67%gap to pasting: -20, -28, -17, -10 points; intervals exclude zero
Oracle (only the right notes), accuracy at every size100%the model reads and combines two notes fine when given them
Single-note questions: retrieval vs pasting100% vs 95-100%
Distractor questions: retrieval vs pasting100% vs 100%
Multi-hop, pasting at 10 / 30 / 60 / 120 notes90% / 85% / 55% / 30%
Multi-hop, any retrieval arm0%, except at 10 notes: top-8 10%, hybrid 30%
Time per question, pasting vs top-3 retrieval, at 10 / 30 / 60 / 120 notes1.5 vs 0.7 s, 4.2 vs 0.8 s, 8.6 vs 0.8 s, 18.4 vs 0.9 spasting was 2.0, 5.5, 10.7 and 20.1 times slower
Folder size at which retrieval catches up on accuracynot reached by 120 notes (about 46K tokens)a straight-line read puts it beyond 120; not run

The short version

When you want an AI to answer from a pile of notes, the standard advice is retrieval: search for the few relevant notes and show only those. The competing approach is to paste the whole folder and let the model sort it out. Retrieval is cheaper and is supposed to win as the pile grows. I tested where. All five use the same setup: the open Qwen3.5-9B model compressed to 4 bits, run through llama.cpp on one 16 GB RTX 5060 Ti, at temperature 0 (the setting that makes a model as repeatable as it gets), with its optional "thinking" mode off and the prompt cache off for every arm. Each experiment's questions, thresholds and analysis were written down before it was run.

  • Pasting the whole folder was more accurate at every size tested, from 10 up to 120 notes (about 46,000 tokens). Retrieval never caught up inside that range.
  • On single-note questions retrieval tied pasting exactly (100% against 95 to 100%), and it was between 2 and 20 times faster.
  • The entire accuracy gap was one question type: needing two notes. Retrieval scored 0% there except at 10 notes, where top-8 scored 10% and hybrid 30%. Pasting scored 90% at 10 notes, falling to 30% at 120.
  • The model is not the problem on multi-hop. Given only the two right notes, it scored 100% at every size. What fails is finding the second note.

Takeaway: pick by the kind of question, not the size of the pile. If questions are mostly "what does note X say", retrieval is as accurate and far faster. If answers need joining two notes, plain retrieval of the closest matches misses the second one, and pasting the lot is better until the pile gets very long.

What the folder looked like

Synthetic project notes for vendors, five notes per vendor, so the folder has 10, 30, 60 or 120 notes of about 300 words each. Each size gets 60 questions:

Question typeWhat it needsWhy it is there
Singleone fact from one notethe plain lookup case
Distractorone fact, with a near-duplicate note from another vendor in the folderdoes the model or the search grab the wrong twin
Multi-hopa fact from an item note combined with one from the vendor-profile notethe answer needs two notes, and the second shares no words with the question

Answers are single values with exact-match checking. Retrieval used a standard embedding model, and one arm combined it with keyword search by merging the two rankings.

Results

NotesPastedTop-3Top-8Top-3 + keywordOracle
100.970.670.700.771.00
300.950.670.670.671.00
600.830.670.670.671.00
1200.770.670.670.671.00

The gap between the best retrieval arm and pasting narrows as the folder grows (from 20 points at 10 notes to 10 at 120) but is still outside the noise at 120. A straight-line read would cross somewhere past 120 notes. I did not run that.

Time per question, mean: pasting 1.5, 4.2, 8.6 and 18.4 seconds at 10, 30, 60 and 120 notes; top-3 retrieval 0.7 to 0.9 seconds at every size, including the embedding call. The prompt cache was off for every arm, so pasting pays full price on each question. With a warm cache (part 1) pasting would look better.

Why multi-hop is the whole gap

The vendor profile note shares no words with the question, making it hard for these retrieval methods to find the second note. At 10 notes, top-8 and hybrid still answered 2 and 6 of 20 multi-hop questions correctly; every other retrieval cell scored zero. So this is partly a design property of the test: questions where the connecting note is invisible to the search. A two-step retriever (find the item note, then search for the vendor it names) would probably fix it. I did not test one.

Pasting's own decline is real but different. At 120 notes, the model fails to combine two notes buried in a 46K-token prompt, 30% right, even though it scores 100% when handed the same two notes. That is a long-context combination failure, not a reading failure.

What I would not claim

  • Synthetic, templated notes with unique values. Real notes are messier, with duplicate facts and vague references.
  • One embedding model, one answer model, one folder per size.
  • Single and distractor questions are at the ceiling for every arm, so they separate time but not accuracy.
  • Sixty questions per size and twenty per type. Per-type numbers are coarse.
  • Cache off is the worst case for pasting. In practice, the 9B model's cache (part 1) would cut its time.

Part 4 of 5. Previous: a stated rule and the same rule implied. Next: what survives when an AI compresses its own context.

Does a company-document assistant need a search system first?

For a small document collection, supplying the whole collection is worth comparing with searching for selected passages. Here it was more accurate but slower, and the search setups missed answers requiring two notes together. This was a synthetic collection of at most 120 notes, not an enterprise archive. Include questions that require both a general policy and an exception in your own test.

Cite this research

Bart Porcincula. "Paste the Whole Folder or Retrieve the Relevant Notes? On a Small Model, Pasting Won Up to 120 Notes." Published 2026-10-30; updated 2026-11-08. https://porsync.com/research/paste-the-whole-folder-or-retrieve-the-notes-small-model

Permanent article URL