RAG that respects access control

A retrieval prototype built on a folder of documents will demo beautifully and then meet its first real corpus. In that corpus, a salary review sits beside a policy document, a board paper beside a product sheet, and half the material is visible to a few hundred people rather than to everyone.

At that moment the interesting question stops being retrieval quality. It becomes whether the system can answer without revealing something the person asking is not allowed to see.

Why filtering after retrieval fails

The tempting shortcut is to retrieve broadly, then drop the chunks the user cannot access before assembling an answer. It is easy to build and it leaks.

• Relevance ranking has already been computed over documents the user cannot see, so the results they do get are shaped by material they have no right to
• Any summary, count, or aggregate computed before the filter carries information across the boundary
• A near-duplicate of a restricted document, held in an accessible location, surfaces content that was supposed to be controlled
• The failure is silent. Nothing errors, and nobody discovers it until an unrelated audit

Access control has to constrain the candidate set before ranking, not after.

Four ways to do it properly

Which one fits depends on how permissions are shaped in the source systems, and how often they change.

Pre-filtering by permission attributes is the common choice. Every chunk carries the access identifiers from its source, and the query filters on the caller's expanded group membership before the vector search runs. It suits SharePoint and file shares, where permissions attach to containers and change rarely.

Index partitioning gives one index per security boundary. Coarser, and it duplicates content that spans boundaries, but it is easy to reason about and easy to prove. It works well where the organization has a small number of hard walls, such as separated legal entities.

Late binding queries the source system for authorisation at retrieval time rather than trusting indexed metadata. Correct by construction, and slow. Reserve it for small, high-sensitivity corpora.

Restricting ingestion to material a defined population may all see avoids the problem instead of solving it. Weaker coverage, and often the right first release.

The part everyone underestimates

Permissions change, and an index is a copy. Someone leaves a project, a document is reclassified, a group is restructured, and the index still holds the previous state until it is told otherwise.

Full reindexing on a schedule is not a control, because the gap between runs is exactly the window that matters. Treat permission changes as events to consume, hold a maximum staleness the business has agreed to, and be able to state what it is. If nobody can say how long a revoked permission takes to reach the index, that number is unbounded.

An index is a copy of your documents and a copy of the permissions that governed them at the moment it was built. Only the first is usually treated as needing maintenance.

What to test before launch

• A restricted document that a test user cannot open in the source system does not appear, quote, or influence an answer
• Revoking access reaches the retrieval path within the agreed staleness window, measured rather than assumed
• Aggregate questions across a mixed corpus do not leak counts or summaries from inaccessible material
• Every answer names its sources, and each named source is one the user can open

What we recommend

Design the permission model before the retrieval model. Pre-filter on expanded group membership for most corpora, partition where there are hard walls, and keep late binding for the small sensitive set.

If the access model is not clear in the source systems, fix that first. A retrieval system built over unclear permissions will make the ambiguity visible at speed and at scale, which is a bad way to discover it.

Why is post-retrieval filtering not enough for RAG?

Because ranking has already run across documents the user cannot see, so their results are shaped by restricted material, and any aggregate computed before the filter carries information across the boundary. Access control has to constrain the candidate set before ranking.

How do you keep an index in step with changing permissions?

Consume permission changes as events rather than relying on scheduled reindexing, and agree a maximum staleness with the business. Scheduled full reindexing leaves exactly the window that matters uncovered. If nobody can state how long a revocation takes to reach the index, it is unbounded.

Should we index everything or only what is broadly readable?

Restricting ingestion to material a defined population may all see is a legitimate first release. Coverage is weaker, but it removes an entire class of failure while the permission model is being worked out, and it can be widened later.

What should be tested before a retrieval system goes live?

That a document a test user cannot open never appears, is quoted, or influences an answer; that revocation propagates inside the agreed window, measured rather than assumed; that aggregates do not leak from inaccessible material; and that every cited source is one the user can actually open.