Why Your RAG Pipeline Keeps Leaking Data It Shouldn’t: Access Control at the Retrieval Layer

This isn’t a hypothetical. It’s the single most common architectural mistake in enterprise RAG deployments, and it’s almost never caught in a demo. Demos use one document set and one user. Production has HR records, legal contracts, board decks, and 200 employees — and the moment two of those employees have different permissions on the same folder, most RAG pipelines quietly stop enforcing anything at all.

The gap: permission-aware UI, permission-blind retrieval

Here’s the pattern that keeps showing up:

1. A company builds a RAG chatbot over its internal document store (Google Drive, SharePoint, Confluence, a document management system).
2. The chat interface correctly checks the user’s identity and shows a nice, permissioned document list.
3. But the vector index behind the chatbot was built once, from everything — HR, finance, legal, engineering — with no per-chunk record of who’s allowed to see it.
4. A user asks a question. The retrieval step searches the entire index for the closest semantic match, finds a chunk from a document they were never granted access to, and hands it straight to the LLM as context.
5. The LLM, doing exactly what it’s supposed to do, answers the question using that chunk — including specifics the user should never have seen.

No error. No audit flag. The interface never even shows the source document, so nobody notices the retrieval crossed a permission boundary — until it shows up as a finding in a security review, or worse, a regulator’s inquiry.

This is why "we already have SSO and role-based access on the app" is not the same claim as "our RAG pipeline enforces access control." SSO controls who can log in. RBAC on the app controls which screens they can see. Neither one touches what happens inside the retrieval step, where a semantic similarity search doesn’t know or care whose document it’s reading from.

Why this doesn’t show up in testing

Three reasons access-control gaps in RAG survive all the way to production:

• Single-tenant test data. Most pilots run against a demo corpus with no sensitivity tiers, so there’s nothing to leak.
• Single-user testing. One developer, one admin account, testing happy paths. The failure only appears once a second user with narrower permissions starts asking questions.
• The LLM sounds confident either way. A permission leak doesn’t look like an error — it looks like the system working well. The chatbot answered accurately. That’s the problem.

Three retrieval-layer patterns that actually enforce permissions

1. Metadata filtering at query time

The minimum viable fix: tag every chunk with the access-control metadata it inherited from its source document, and apply that as a hard filter inside the similarity search — not after.

Building on the document_chunks schema from our pgvector tutorial, extend it with an access table:

-- Groups/roles allowed to see each document
CREATE TABLE document_acl (
    document_id  BIGINT REFERENCES documents(id) ON DELETE CASCADE,
    principal    TEXT NOT NULL,   -- e.g. 'role:finance', 'user:alice@company.com', 'group:leadership'
    PRIMARY KEY (document_id, principal)
);

CREATE INDEX ON document_acl (principal);

def semantic_search_with_acl(
    query: str,
    user_principals: list[str],   # e.g. ["user:alice@company.com", "role:finance", "group:apac"]
    top_k: int = 5,
) -> list[dict]:
    query_embedding = get_embedding(query)
    conn = psycopg2.connect(DB_URL)
    cur = conn.cursor()
    try:
        cur.execute(
            """
            SELECT dc.id, dc.content, d.title, d.source_url,
                   1 - (dc.embedding <=> %s::vector) AS similarity
            FROM document_chunks dc
            JOIN documents d ON d.id = dc.document_id
            WHERE EXISTS (
                SELECT 1 FROM document_acl a
                WHERE a.document_id = d.id
                AND a.principal = ANY(%s)
            )
            ORDER BY dc.embedding <=> %s::vector
            LIMIT %s
            """,
            [query_embedding, user_principals, query_embedding, top_k],
        )
        return cur.fetchall()
    finally:
        cur.close()
        conn.close()

The filter runs inside the query, not as a post-hoc check on returned results. That distinction matters: filtering after retrieval means you fetched the forbidden content into your application’s memory before deciding to discard it — the leak already happened at the network layer, and a bug in the post-filter step means the content reaches the LLM anyway.

2. Row-level security as a backstop

Application-layer filtering is only as reliable as every code path that queries the table. Any script, admin tool, or future integration that forgets the WHERE EXISTS clause reopens the hole. Postgres Row-Level Security (RLS) enforces the same rule at the database layer, so it applies even to queries the application developer didn’t anticipate:

ALTER TABLE document_chunks ENABLE ROW LEVEL SECURITY;

CREATE POLICY chunk_access_policy ON document_chunks
USING (
    EXISTS (
        SELECT 1 FROM document_acl a
        WHERE a.document_id = document_chunks.document_id
        AND a.principal = ANY(current_setting('app.user_principals')::text[])
    )
);

# Set the session's principals before running any query in this connection
cur.execute("SET app.user_principals = %s", [user_principals])

This is defense in depth, not a replacement for the application-layer filter — it’s what stops a permission bug in one endpoint from becoming a data breach across the whole system.

3. Handling permission changes and stale embeddings

Chunks and embeddings are static once written. Permissions are not — someone leaves the finance team, a contract gets reclassified, an offboarded contractor’s access should vanish immediately. Two things need to happen:

• ACL sync, not content re-embedding. When permissions change, update document_acl directly. You don’t need to re-embed the document — the access-control table is separate from the vector data, so revoking access is a metadata write, not a pipeline re-run.
• A sync job from your source of truth. If documents live in Google Drive or SharePoint, run a scheduled job that pulls current folder/file permissions and reconciles document_acl against them — ACLs drift from source systems faster than most teams expect, particularly with nested folder inheritance.

Architecture overview

flowchart TD
    A["User query + identity token"] --> B["Resolve user principals\nroles, groups, direct grants"]
    B --> C["Embed query"]
    C --> D["Filtered similarity search\nACL check inside the query"]
    E["document_acl table"] --> D
    F["Source system permission sync\nDrive, SharePoint, DMS"] --> E
    D --> G["Permitted chunks only"]
    G --> H["LLM generates answer"]
    H --> I["Retrieval audit log\nwho, what, when, matched principal"]

Auditability: the part compliance teams actually ask for

A permission-aware retrieval layer gives you something a permission-blind one can’t: a defensible log. Every retrieval event should record the querying identity, the principals resolved for that identity, and which documents were eligible to be returned — whether or not they were the top match. When a security review or a regulator asks "how do you know your AI system didn’t expose restricted data," this log is the answer, not a promise.

FAQ

Isn’t this what the front-end permission check already handles?

No. The front-end check controls what a user can click on. It has no visibility into what the retrieval step pulls into the LLM’s context window internally. Those are two separate enforcement points, and only one of them is usually built.

Does this slow down retrieval?

An indexed EXISTS filter against document_acl adds low single-digit-millisecond overhead when document_acl.principal is indexed, which is negligible next to embedding generation and LLM inference time.

What if permissions are more complex than flat roles — nested folders, inherited groups, temporary access grants?

Model each inheritance path as a row in document_acl at sync time rather than trying to compute inheritance at query time. Flatten the hierarchy during the sync job; keep the query itself a simple set-membership check.

Do we need this if our documents aren’t that sensitive?

If the corpus mixes any tier of sensitive material — HR, legal, financial, customer PII — with general content, yes. The failure mode isn’t "occasionally embarrassing"; it’s a disclosure that’s hard to explain after the fact because nothing in the system flagged it as unusual.

Building or auditing a RAG pipeline that needs to enforce real access control — not just a permissioned UI in front of an unpermissioned index? Contact us at hello@simplico.net — we design and deploy production RAG architectures for enterprise clients across Thailand, Japan, and Southeast Asia.