Blueprint · Reference Architecture
Solution Blueprint: Enterprise RAG Reference Architecture
Production RAG that survives contact with the enterprise: ACL-aware retrieval, structure-aware chunking, eval-gated releases, ingestion SLAs, and the governance controls that make grounded AI defensible.
What this is: an illustrative reference blueprint, not a client engagement. Scale figures are scenario assumptions, labeled as such. We never invent client names, borrow results, or imply an engagement that didn't happen.
00 · ASSUMPTIONS
Everything this blueprint takes for granted — stated
- Use case
- Grounded Q&A over private enterprise content (policies, procedures, knowledge bases) for employees with differing access entitlements — not public chat, not autonomous agents.
- Model posture
- Frozen foundation models via API or self-hosted; no enterprise content used for training or fine-tuning. RAG over frozen models only.
- Access model
- Per-user entitlements enforced at retrieval time via ACL filters in the index — assumed mappable from source-system permissions.
- Content operations
- Every corpus has a named owner, a review cadence, and a freshness SLA; ingestion is a pipeline, not a one-time upload.
- Eval discipline
- A golden question set with expected citations exists before launch and gates every change to index, prompts, embeddings, or models.
- Answer contract
- Substantive claims carry citations; the system abstains when evidence is thin rather than generating plausible text.
- Scale (scenario)
- Illustrative: a multi-million-chunk corpus, thousands of daily users, p95 answer latency in single-digit seconds — sized to force real architectural choices.
- Governance
- Model, prompt, and embedding versions registered and change-controlled; queries, retrieved chunks, and answers audit-logged.
- Risk
- Top risks: stale content (freshness failures), entitlement-mapping errors, chunking that splits meaning, and silent retrieval-quality regression.
01 · PROBLEM
The demo worked; production is a different country
Enterprise RAG demos are easy: upload documents, ask questions, get fluent answers. Production is where it breaks — retrieval quality decays silently, stale documents produce confident wrong answers, access control is an afterthought, and nobody can say what "good" means because there's no eval set. The result is a system everyone demos and nobody trusts.
This blueprint designs the boring parts that make RAG trustworthy: the ingestion pipeline with SLAs, access control at retrieval, the eval harness as release gate, and the governance that lets a security review say yes.
02 · SCALE
Illustrative scale (scenario assumptions)
- Corpus
- Illustrative: low millions of documents → tens of millions of chunks, continuously updated by many content owners.
- Users
- Illustrative: thousands of employees daily, with sharply different access entitlements across roles and departments.
- Latency
- Illustrative: p95 answer latency in single-digit seconds — fast enough for in-workflow use, honest about the retrieval + rerank + generation chain.
- Freshness
- Source-dependent SLAs: some corpora hourly, some weekly — the ingestion monitor enforces per-source freshness, not a global average.
- Change rate
- Index, prompt, embedding, and model changes are frequent; every one passes the eval gate — which is why the gate must be fast and automated.
03 · CONSTRAINTS
What the design had to respect
- Access control correctness: users must never see content they're not entitled to — enforced at retrieval, not as answer post-filtering.
- Grounding: citations on substantive claims; abstention on thin evidence. Fluency without grounding is a liability.
- Freshness: stale answers are worse than no answers; ingestion SLAs and overdue-source alerting are first-class.
- No training on enterprise data: content stays in the index where access control and updates work — never baked into weights.
- Auditability: every answer traceable to its query, retrieved chunks, and the configuration that produced it.
- Cost sanity: reranking and generation are the expensive steps; the architecture must not rerank the world per query.
04 · ARCHITECTURE
The pipeline, the platform, and the gates
Ingestion is a pipeline with SLAs: connectors pull from sources, parsing handles structure and sensitive data (de-identification, metadata attachment — owner, version, effective date, entitlement tags), structure-aware chunking respects document boundaries, and versioned embedding models produce the vectors. An ingestion SLA monitor alerts on overdue source refreshes — the freshness control.
The retrieval platform centers on a vector index carrying ACL filters: retrieval only ever sees what the asker may see. Query understanding (rewrites, filters, routing) feeds hybrid search (dense + keyword) and a reranker; a prompt and configuration registry versions everything; and the eval harness — golden questions with expected citations — gates every change.
Serving orchestrates the frozen model, assembles citations per claim, applies guardrails (abstention, PII filtering), and writes the full audit log. The data flow is a loop: audit and eval results feed content operations and chunking improvements.
05 · DECISIONS
Key tradeoffs
| Decision | Chosen | Why |
|---|---|---|
| Grounding approach | RAG over frozen models, not fine-tuning | Fine-tuning bakes content into weights — un-auditable and un-revocable. RAG keeps content in the index where access control and updates work. |
| Access control | ACL filtering at retrieval time | Post-filtering answers leaks via phrasing and timing. If the retriever can't see it, the model can't reveal it. |
| Vector store | Start with Postgres + pgvector; move to dedicated when justified | pgvector is legitimate into the single-digit millions of vectors at modest QPS. Dedicated stores earn their place on hybrid search at scale, multi-tenancy, or strict p99 latency. |
| Chunking | Structure-aware, per document type | Token-count chunking splits procedures mid-step and tables mid-row — the source of confident wrong answers. Boundaries follow document structure. |
| Abstention | Required on thin evidence | An abstaining system routes questions to humans; a guessing system routes incidents to you. Abstention is the trust feature. |
| Release process | Eval gate on every change | Retrieval quality regresses silently. The golden-question suite is the only thing between an improvement and a quiet degradation. |
06 · IMPLEMENTATION
Phased delivery
07 · SECURITY & GOVERNANCE
Controls
- Sensitive-data handling in ingestion: identification, de-identification where identifiers aren't needed, and metadata tagging for entitlement enforcement.
- Encryption in transit and at rest; least-privilege service identities per pipeline stage; prompts and logs scrubbed before any external exposure.
- Audit logging of queries, retrieved chunks, and served answers — the evidence trail for compliance review and incident response.
- Model, prompt, and embedding version registry with change control; every answer traceable to its configuration.
- Content ownership with review cadence; incident playbook for wrong or leaked answers — containment, root cause, regression test added to the eval set.
08 · RESULTS
Success criteria (illustrative targets, not achieved outcomes)
- Retrieval precision on the golden question set meets the agreed bar before launch — and never regresses past it afterward, because the gate blocks the change.
- Every substantive claim in pilot answers carries a correct citation; abstention rate stays within the tuned band — neither guessing nor refusing everything.
- Adversarial access-control testing finds no entitlement leaks before launch and none under audit after.
- Ingestion freshness SLAs met per source; overdue-source alerts fire before users notice staleness.
These are acceptance thresholds for the scenario design. We publish measured client outcomes only with permission — which is why this page carries the reference-architecture label.
09 · LESSONS LEARNED
What generalizes
- Retrieval quality is the whole game. Chunking, metadata, and reranking decide answer quality long before model choice matters.
- Freshness is a pipeline problem. The first production incidents are almost always stale content — ingestion SLAs beat model upgrades.
- The eval set is the product spec. Golden questions with expected citations are how the organization says what "good" means, in a form engineers can build against.
- Access control at retrieval, or not at all. Entitlements belong in the index.
- Abstention is a trust feature. "I don't have a reliable source for that" earns the trust that lets the system answer the next hundred questions.
10 · TECHNOLOGIES
Representative, not prescriptive
11 · RELATED
Pattern, case study, toolkit, service
Practitioner case study
A healthcare enterprise's RAG platform — the experience this blueprint's controls draw on.
Read case study →RAG accuracy in production
Chunking, hybrid search, reranking, ingestion SLAs, and evals.
Read the guide →Hallucinations are a data problem
Grounding, constrained generation, verification, and abstention.
Read the guide →Enterprise RAG pack
Free working documents: retrieval eval template, chunking decision guide, ingestion SLA worksheet.
Get the pack →Agentic AI practice
AnovaCloud's agentic AI services: RAG, agents, evals, and governance.
Explore practice →Start here
Talk to an Architect
Bring your hardest AI, data, or modernization problem. We'll tell you plainly whether we can help — and what it takes.