Enterprise RAG search: architecture, access control and answer quality

Enterprise RAG is useful because an answer can rely on permitted internal sources and show where the conclusion came from. Without access control and index governance, it becomes a convenient interface to stale or restricted data.

RAG is not one model. It is a pipeline for ingestion, text extraction, chunking, indexing, retrieval, context assembly and answer generation. Every stage affects the final result.

Define one workflow first

Do not begin by loading the complete corporate drive. Choose one process:

• operations manuals;
• support-agent assistance;
• navigation through policy documents;
• technical-document search;
• employee guidance from internal procedures.

Define the audience, approved sources, freshness requirement, citation rules and cases where the system must decline to answer.

Give every source an owner

Before indexing, record:

• source system;
• data type and classification;
• owner;
• validity period;
• access rules;
• update interval;
• retention basis;
• deletion process.

Where several versions exist, the system needs a reliable active-version marker. File modification time alone is not always sufficient.

Carry access control into retrieval

Building one unrestricted index and hiding links in the interface is unsafe. A user can receive content from a restricted document through the model’s answer.

Common designs use either separate indexes for access domains or metadata filtering with document ACLs. In both cases, authorisation runs for every query before context reaches the model.

The indexer’s broad service account must not silently grant every user access to everything it could read.

Prepare documents for retrieval

Chunk size affects both precision and context noise. Preserve:

• heading hierarchy;
• document identifier;
• version and effective date;
• owner;
• ACL;
• source fragment link;
• language;
• content type.

Tables, diagrams and scans need dedicated handling. OCR output requires validation because a mistaken number or negation can change an answer.

Use hybrid retrieval

Vector search captures semantic similarity but can miss exact product codes, agreement numbers and parameter names. Enterprise search often combines:

• full-text search;
• vector retrieval;
• metadata filters;
• reranking;
• a relevance threshold.

When no adequate source exists, the system should say so rather than generate a confident guess.

Make answers verifiable

An answer should point to exact fragments. A user must be able to open the source, see its version and date, inspect the relevant paragraph and report an error.

A citation does not prove that the conclusion is correct, but it makes review possible. For sensitive decisions, RAG should support a human decision rather than replace it.

Treat document content as untrusted

A document may contain text that a model interprets as an instruction to ignore policy, reveal a prompt or call a tool. Keep system instructions, user input, retrieved documents and tool results as distinct trust zones.

Document text must never change permissions or the available tool set. This extends the controls described in secure AI-agent architecture.

Audit and evaluate

Log the request ID, user and access domain, index and model versions, retrieved documents and scores, filters, response, refusal or fallback, user feedback and runtime. Avoid secrets and unnecessary personal data in logs.

Build a test set of real questions with expected sources. Evaluate retrieval, source support, unsupported claims, ACL enforcement, correct refusal and regressions after an index update.

Include ambiguous queries, stale versions, typographical errors, duplicate names and questions with no answer.

Operational checklist

• One workflow and audience are defined.
• Every source has an owner.
• Active and superseded versions are distinguishable.
• ACLs run before context reaches the model.
• Users can open the exact cited fragment.
• Weak retrieval produces a refusal.
• Documents are treated as untrusted input.
• A regression set measures answer quality.
• Withdrawn data is removed from the index.
• Logs do not expose unnecessary information.

How we can help

Logic Telecom can help design ingestion, API integrations, access control and observability for enterprise RAG. Start with a bounded document set whose owner, users and acceptance criteria are known.