RAG (retrieval-augmented generation) is an approach in which a language model grounds its answer in passages retrieved from an organization's own documents. This post describes which decisions the person managing a RAG project should write down early; it is my working method, not a claim about a measured result.
Why is a RAG project a management problem?
In RAG projects the model is often a ready-made service; the real work is in decisions such as which documents will be used, who may see what and how answer accuracy will be measured. If those decisions are not written down, the project ends up at “the demo looks good, but it cannot be trusted live”. The steps below follow the order I use in AI projects.
1. Scope: which questions will it answer?
The first step is to write down which user questions the system will answer and which it will not. “Query all company documents” is a wish, not a scope. Sample user questions are collected; they define both the scope and the later evaluation.
2. Source documents and data ownership
Which documents go into the index? Are they current, are there conflicting versions, do they contain personal data? Each group of documents needs an owner, and how the index is refreshed when a document changes is defined up front. The basis of an answer must be showable: users should be able to see which section of which document an answer was built from.
3. Access control
RAG can leak the content of a confidential document to someone who has no access to it. The access rule is therefore applied at query time and written into the first plan: who may ask about which group of documents? This is an architecture decision, not a feature to add later.
4. The evaluation set
A small evaluation set is prepared from real user questions and the source documents expected for them. Every version is compared on this set; “it looks good” is not enough. Whether the system found the right document and whether it drew the right conclusion from the document it found are examined separately.
5. Pilot, cost and latency
The pilot runs with a limited group of users. Response time and usage cost are measured at this stage; the decision to expand is written down after weighing accuracy, time and cost together.
6. Monitoring and change management
While live, user feedback, unanswered questions and document changes are monitored. When the model, the prompt or the document set changes, the evaluation set is run again. This process overlaps with the lifecycle and monitoring questions in the ISO/IEC 42001 checklist.
Frequently asked questions
What is the difference between RAG and fine-tuning?
RAG grounds the answer in document passages retrieved at query time; when documents change, the index is refreshed. Fine-tuning trains the model itself with examples. The two can also be combined; the choice depends on data freshness, traceability and cost.
What is the most common management mistake in a RAG project?
Leaving the evaluation set and access control to the end. Both affect the architecture, so both belong in the first plan.
How long does a RAG project take?
It depends on the number of documents, the complexity of access control and the integrations; once the scope is written, the pilot duration is planned. I do not promise a duration I have not measured.