When to use retrieval vs pasting documents into chat

When paste is enough

Paste first when:

• The doc fits comfortably in context
• You know which section matters
• The question is one-off
• You need a tight audit trail of exactly what the model saw
• You are still learning the job and do not want retrieval mistakes mixed in

Examples: rewrite one policy page; extract actions from one meeting note; compare two vendor one-pagers you already opened.

Paste is honest. You chose the grounded context.

When retrieval starts to win

Consider retrieval when:

• You cannot remember which of twenty files holds the answer
• The corpus is larger than you want to paste
• Many people ask similar questions against the same library
• You need citations back to source files as a default

Examples: “What did we decide about refunds last quarter?” across a folder of decision logs; support FAQ draft from hundreds of tickets (after cleaning).

Retrieval is a search problem plus a generation problem. If search is wrong, the answer is confidently wrong.

Failure modes to expect

Paste failures

• You forgot the key appendix
• The thread got long and early facts fell out of the effective context
• You put secrets you should not have into the prompt

Retrieval failures

• Wrong neighbor docs ranked high
• Stale duplicates beat the current SOP
• Chunk boundaries split a table or a rule in half
• The model ignores a retrieved warning line

Design for the failure you can catch. Paste is easier to catch by eye. Retrieval needs spot checks and clean corpora.

A practical decision rule

Ask:

1. Do I already know the file?
2. Is it small enough to paste without pain?
3. Is this a repeated question type?

If yes/yes/no → paste.
If no/—/yes → retrieval (or folder chat).
If the corpus is a mess → clean files before either path; see the hygiene steps in the personal knowledge-base guide.

How to paste well

• Provide the minimum section, not the whole binder
• Put the question after the text
• Say “use only this text; if missing, say [NEED]”
• Start a new thread when the topic changes

How to try retrieval without a science project

1. Curate one folder of current docs only
2. Use a product’s built-in “chat with files / project” feature if you have one
3. Ask questions that should cite a known file; verify the citation
4. Keep a dead-letter list of questions that failed; fix files or chunking later

Do not start by building a custom vector index unless you already know paste and folder chat are not enough. Off-the-shelf vs custom still applies.

Worked example (pattern only)

Question: “What is our on-call handoff checklist?” You know it lives in sop/oncall.md (two pages). Paste that file.
Question: “Summarize every pricing decision in 2026.” You have twelve dated decision logs. Use folder retrieval or open them in a project chat—and demand file citations.

Cost and complexity

Paste costs attention and occasional re-uploads. Retrieval costs setup, re-indexing, and debugging bad hits. For a five-person team, paid complexity should follow pain you can name weekly—not a blog post about embeddings.

How a five-person team runs this week

Pick five real questions from the last month. For each, try paste first if you know the file. Log which ones needed search across many files. Only those justify a folder-chat or retrieval trial. Clean filenames before you turn search on. If retrieval cites a stale doc, archive the stale doc the same day—do not “tune the model” to ignore garbage you left in the folder.

Short close

Paste when you know the source; retrieve when search is the real job—and clean the corpus either way.