Retrieval and RAG: useful, leaky if unbounded
Retrieval makes answers better by fetching context. Unbounded retrieval makes the model a search engine over everything it should never have seen.
Retrieval makes answers better by fetching context. Unbounded retrieval makes the model a search engine over everything it should never have seen. RAG is a data path with a query. Usefulness and leakiness share the same pipe. Bound the corpus, the query, and the returned chunks — or you rebuilt shadow search.
---
1. Opener: Fetch-then-generate, not magic memory
RAG is not magic memory. It is fetch-then-generate.
Teams hear “grounded answers” and imagine a model that somehow already knows the wiki. What actually happens is plainer: something is indexed; a query runs; top-k chunks are packed into a prompt; then the model generates. The weights may never have trained on your corpus. The path still fetched those chunks into context.
That is why this primer sits in AI × Data Security, not in “prompt craft.” Usefulness and leakiness share the same pipe. The same machinery that reduces hallucination by fetching facts can also pull neighboring secrets, ACL-hidden text, and file paths the UI never showed. “Just add RAG” is not a free upgrade. It is a decision to open a search-shaped data path.
This piece explains the pipe and the three bounds. It does not sell a suite. Named platforms can appear later when a Learn-backed fact is required. They are not the opener and not the product being sold here.
If you already sorted paste / connect / retrieve / remember under one assistant face (#12), this primer deepens retrieval. If you already treated a prompt as a shipping label (#13), this is what happens when the data was fetched instead of pasted.
---
2. Mechanik: Index → query → top-k chunks → prompt
Name the stages in order. Each stage can over-share.
Index
Someone chose a corpus and built an index — embeddings, keyword, hybrid, or a product wrapper that hides the choice. Whatever was eligible at index time is what retrieval can later see. Soft UI permissions on the source system do not automatically survive the embed step. If a document was readable to the indexer identity, its chunks may live in a store that answers queries without replaying the original ACL check — unless you designed for that check.
Indexing is already movement: content left its home store and entered a retrieval surface.
Query
A user question, a rewritten query, a tool argument, or an agent step becomes a search. The query decides which neighborhood of the index gets pulled. Over-broad queries, synonym expansion, and “helpful” rewrite that adds related terms can widen the pull beyond the task. The query is not innocent metadata. It is the steering wheel on the data path.
Top-k chunks
Retrieval returns a short list of passages. Those passages are the payload. Chunk boundaries matter: a chunk can include a header, a path, a neighboring paragraph, a table cell, or a comment the human never meant to surface. Top-k is a dial, not a guarantee of need-to-know. Raising k to “improve quality” is also raising how much foreign context enters the prompt.
Prompt
Chunks are concatenated (with or without citations) into the model context. From here, the familiar paste residues apply (#13): content in the prompt, transit and logs you do not fully control, copies that outlive the answer. Retrieved context is still prompt content. It is just content the human did not type.
A useful operator table:
| Stage | What moves | Dominant over-share risk |
|---|---|---|
| Index | Corpus → retrieval store | Over-broad eligibility; ACL loss at embed time |
| Query | Question → search | Pulls neighboring or unrelated sensitive hits |
| Top-k chunks | Hits → candidate context | Chunk includes hidden neighbors, paths, headers |
| Prompt | Chunks → model input | Same residues as paste; citations that leak location |
Generation quality is a reliability story. What entered the prompt from the index is a data-path story. Do not “fix” an over-fetch with a friendlier system prompt.
---
3. Failure modes: when the pipe becomes shadow search
Unbounded retrieval is not a dramatic breach headline. It is quieter: the assistant becomes a search engine over everything it should never have seen — with a chat face that makes the pull look like helpfulness.
Over-broad corpus
If the index includes “everything the bot identity can read,” you did not design a knowledge base. You designed a secondary search surface. Whole-drive dumps, ticket archives, and HR folders mixed in “for completeness” are classic widenings. The failure is eligibility, not model cleverness.
Queries that pull neighboring secrets
Semantic search finds nearby meaning. Nearby is not authorized-for-this-task. A customer question can surface an adjacent note with credentials or another customer’s thread. Keyword search does the same with shared IDs. “Helpful” query rewrite that broadens terms widens the neighborhood.
Chunks that include what the UI hid
Source UIs hide fields by role or redaction. Chunkers often do not. A chunk may carry ACL-hidden neighbors, comments, or paths the screen never showed that user. Without identity re-check at fetch time, the chat can show what the portal withheld.
Citations that reveal paths
Even harmless-looking citations, titles, and breadcrumbs can reveal where files live or that a document exists. Existence and location are already information. Treat citations as part of the return bound, not decoration.
None of these require model weights trained on your data. Blast radius changed because chunks moved into context — the done-when, said early so controls have a clear target.
---
4. Three bounds — or you rebuilt shadow search
Bound three things. Miss one and the other two do not save you.
Corpus scope
What is eligible for the index? Name collections and systems in plain sentences. Prefer task corpora over “all company content.” Separate highly sensitive stores until there is need and a control story. Re-index is a change event: new folders in scope are new blast radius.
Corpus scope answers: what could ever be fetched?
Query bounds
What may be asked, by whom, against which index? Identity-aware retrieval belongs here: the query runs as a principal; results filter to what that principal may see — not only what the indexer once saw. Allow-listed indexes per workspace and refusing cross-team queries are query bounds. Soft language (“searches your knowledge”) is not a bound until the principal is named.
Query bounds answer: what searches are allowed to run?
Return / chunk bounds
What may enter the prompt? Cap k for the task; refuse chunks that fail a sensitivity check; redact before embed when the source cannot be narrowed; decide whether citations show path, title, or neither. Fetch logging is part of the return story: without chunk IDs in the log, over-share investigation is theater.
Return bounds answer: what is allowed to become prompt content?
A one-line operator test: If this index, this query, and this top-k were a human search box, would we call that box in-policy? If not, the chat face does not make it in-policy.
---
5. Controls (conceptual — no product laundry)
Controls are operator moves, not a feature matrix. claimState stays promise: describe the class of control; do not invent vendor capabilities or metrics.
- Corpus scope — explicit eligibility; separate sensitive collections; treat re-index as a change with owners.
- Identity-aware retrieval — fetch as a named principal; re-check authorization at query time when the design allows; avoid “bot can read everything, users ask through the bot.”
- Redaction before embed — remove secrets, credentials, and needless identifiers at index time when the source cannot be narrowed. Embed is a copy; treat it like one.
- Logging of what was fetched — which index, which query (or hash), which chunk IDs, which principal, which session. Without fetch logs, revoke and investigation are theater.
- Kill-switch on indexes — disable an index, quarantine a collection, revoke the retrieval connector mid-path. If kill only works after a long vendor ticket, you do not have mid-path control. Mark uncertainty as promise until you can demonstrate the kill.
These rhyme with the four questions from the data-path field note (#12): what left (chunks into prompt), what persisted (index + logs + thread), who can trigger (principals and shared workspaces), can you revoke mid-path (index kill / connector revoke). Retrieval does not retire those questions. It specializes them.
Purview-no-Suite-Sell stays in force. When data platforms enter later pieces, frame govern / protect / manage and Docs-loyal names. Do not open with laundry lists, license SKUs, or leaderboard numbers. Buying a portal does not retire the three bounds.
---
6. Link to #12 and #13: still a path, sometimes safer than paste
#12 — When an AI tool becomes a data path. Retrieved context is still movement. Who triggered the query, what chunks left the index, what was stored in the thread or fetch logs, what can be replayed by re-running the same search — those are path properties under a retrieval face. The session-boundary test still applies: did anything leave (or enter) that the sealed chat did not already hold?
#13 — Putting business data into a prompt. Sometimes retrieval is safer than pasting a whole file into the box: minimum needed chunks beat a full dump “for context.” Safer is not free. The residues move from “what I typed” to “what the index returned.” The shipping-label habit remains: sensitivity of the content, path of the request, residue after the answer. Red lines still hold — secrets and credentials should not ride in chunks any more than they should ride in paste.
A practical sorting line for pilots:
| Choice | When it helps | What you still owe |
|---|---|---|
| Paste only | Small, deliberate excerpts; no index yet | Checklist from #13; know retention |
| Bounded RAG | Task corpus; identity-aware fetch; capped return | Three bounds + fetch logs + kill-switch |
| Unbounded “search everything” | Almost never for sensitive estates | You rebuilt shadow search with a chat UI |
Do not invent a percentage of “how much safer.” Literacy, not theater.
---
7. Done when: blast radius without training on the data
Done when a reader can explain — in a few plain sentences — why “just add RAG” changes the blast radius even when the model weights never trained on that data.
The short form:
- RAG is fetch-then-generate, not magic memory.
- Index → query → top-k chunks → prompt; each stage can over-share.
- Failure modes include over-broad corpus, neighboring secrets, ACL-hidden chunk content, and citations that leak location.
- Bound corpus, query, and returned chunks — or you rebuilt shadow search.
- Retrieved context is still a data path (#12); sometimes safer than paste (#13), never free.
If those five hold without a suite pitch, the primer did its job.
---
8. What this piece is not
Clear negatives save time — outline non-goals, said as operator copy:
- Not a vendor bake-off. No suite SKUs, no feature laundry, no “which platform wins RAG.”
- Not a vector-DB tutorial. Embeddings are a mechanism; the security story is eligibility, query principal, and return bounds.
- Not invented metrics. No fake groundedness percentages, no “X% fewer hallucinations,” no traction theater. claimState stays promise.
- Not a claim that RAG is bad. Bounded retrieval is useful. Unbounded retrieval is leaky. The title is the thesis.
- Not fear marketing. Risks stay sober and paired with bounds and kill-switch language.
- Not an agents deep dive. Agent loops that keep retrieving get their own field note (#15). Here: even a single retrieval hop is already a data path.
Platforms — including Microsoft Purview and Docs concepts such as govern, protect, and manage — can appear when a Learn-backed fact is required. Order stays: path questions first; named controls only when a practice question needs them. kombify is not the subject of this line.
---
9. How this fits the reading path
Arc for the blog: Early Foundations → Data Security → AI × Data Security.
Current Lesepfad preference: S3-first if you already live with assistants and connectors. This primer is the retrieval deep-dive (#14) after #12 and #13.
Light pointers (titles, not invented URLs):
- #12 — When an AI tool becomes a data path
- #13 — Putting business data into a prompt: a practical risk checklist
- #15 — Agents that can act: blast radius before autonomy theater
- #16 — AI Security as a reading path: foundations → data → intersection
Rails stay navigation: this piece is a primer. Pick the next article from the path; do not shop a suite.
---
10. Close: Same pipe, three bounds
Retrieval makes answers better by fetching context. Unbounded retrieval makes the model a search engine over everything it should never have seen.
RAG is fetch-then-generate: index, query, top-k chunks, prompt. Usefulness and leakiness share the same pipe. Bound the corpus, the query, and the returned chunks. Identity-aware retrieval, redaction before embed, fetch logging, and a kill-switch on indexes are controls — not a product pitch. Retrieved context is still a data path. Sometimes safer than paste. Never free.
Done when a reader can explain why “just add RAG” changes blast radius even when model weights never trained on that data — without a feature matrix, without a suite pitch, and without inventing a score.
Status: draft / PROMISE. No publish. No Live wire from Caramba. No suite sell. No invented metrics. Review: kombinator2.
---
Series path: S3 AI × Data Security · slot #14 · rail primer. Companion spine: #12 data path · #13 paste checklist. Next: #15 agents / blast radius.
Conversation
No comments yet.