Question 1

Why use a requirements template before building a RAG system?

Accepted Answer

Most RAG projects that ship and then fail were never written down. They were built from a Tuesday-afternoon demo, scaled to a Friday production deploy, and then quietly broke when the document set grew, the ACL story got complicated, or the citation accuracy turned out to be 70 percent. A written spec forces the seven hard decisions early: sources, ACL, chunking, embeddings, retrieval, citations, fallback. Each one has a cheap answer at week one and an expensive answer at week twelve.

Question 2

What is the minimum viable RAG specification?

Accepted Answer

Five sections. Document sources with refresh cadence. Permission model (per-document, per-folder, or none). Chunking strategy (size, overlap, semantic or fixed). Retrieval (top-k, hybrid search yes/no, reranker yes/no). Evaluation set (20 to 50 questions with expected sources). Everything else can be defaulted, but these five must be written down. Without them, you cannot answer 'is the retrieval good enough' for anyone.

Question 3

How do I evaluate a RAG system?

Accepted Answer

Build an eval set of 20 to 50 question-answer pairs from real users (not from the indexed docs). Measure two things separately: retrieval recall (does the right chunk appear in the top-k) and answer faithfulness (does the response stay grounded in the retrieved chunks). 80 percent retrieval recall and 90 percent faithfulness is a reasonable production floor. Both need to hold up after each ingest or model change &mdash; gate releases on the eval, not on developer judgment.

Question 4

What chunking strategy should I use?

Accepted Answer

Start with 500 to 800 tokens per chunk and 10 percent overlap. Use semantic boundaries (headings, paragraphs) over fixed character counts when the source structure is clean. Smaller chunks (200 to 400 tokens) help when answers are sentence-level facts; larger chunks (1,000 to 1,500) help when context coherence matters more than recall. Re-chunking is cheap if you store source text alongside chunk text &mdash; do that from day one.

Question 5

When does RAG fail and how do I plan for it?

Accepted Answer

RAG fails when the user asks a question the index cannot answer, when retrieval returns plausible-but-wrong chunks, when permissions get bypassed via clever prompting, or when the LLM ignores citations and makes up sources. Plan three fallbacks: a no-results path (say 'I cannot find this in the docs' rather than hallucinate), a low-confidence path (return retrieval results without LLM synthesis), and an escalation path (route to a human or a web search tool). Write these into the spec, not into the prompt as an afterthought.

RAG-System-Anforderungstemplate RAG System Requirements Template

ErforderlichRequiredDie 5 PflichtfelderThe 5 required fields

Quellen-DetailsSource details

ACL-DurchsetzungACL enforcement

Chunking

Embedding & IndexEmbedding & index

Retrieval & RerankingRetrieval & reranking

Citation-FormatCitation format

Fallback-VerhaltenFallback behavior

Evaluation & SLOsEvaluation & SLOs

Monitoring & OpsMonitoring & ops

Generierte SpecGenerated spec

RAG-Implementierung in 3 WochenRAG implementation in 3 weeks

Wie dieses Template zu nutzen istHow to use this template