In an enterprise RAG implementation, success depends on much more than whether the solution includes a vector database. The organisation must limit retrieval to a well-defined business scope, preserve source-document permissions at query time, evaluate retrieval and answer quality separately, and make document updates, auditability and safe refusal part of daily operations. This practical guide helps companies in Thailand and across ASEAN assess RAG development consistently—from the RFP and proof of concept (PoC) through security, cost estimation and acceptance testing. Its central recommendation is simple: do not begin with an enterprise-wide rollout. Start with one business process and one document collection with a clear permission boundary.
What to decide before starting an enterprise RAG implementation
Retrieval-Augmented Generation (RAG) retrieves information relevant to a user’s question from sources such as internal documents, then supplies that evidence to a generative AI model to produce an answer. It is well suited to generative AI with internal data because it can reference corporate policies, product specifications, work instructions and previously approved materials that were not part of the model’s original training data.
RAG is not, however, a box that automatically produces correct answers as soon as documents are uploaded. Microsoft identifies query understanding, access to multiple sources, token constraints, response time, and security and governance among the challenges of RAG. Google Cloud similarly recommends establishing a baseline evaluation and then iterating on source data, layout parsing, chunking and query refinement. Enterprise RAG development is therefore a business-improvement programme that must integrate retrieval, generation, permissions and operations.
At the first planning meeting, decide these four points before discussing products:
- Which business decision, performed by which users, will the system support?
- Which documents are authoritative, and who is accountable for them?
- Who is permitted to view each document?
- What constitutes PoC success, and which events require the project to stop?
“Enable every employee to search every internal document in natural language” may sound clear, but it expands the scope while leaving the business use case, permission boundary, definition of correctness and update ownership unresolved. A better initial PoC scope would be, for example, “enable the quality department to search approved quality procedures.” The user group, purpose, corpus and owner should fit in one sentence. This is an example of appropriate design granularity, not a claim about a specific customer implementation.
RAG, fine-tuning and simple long-context uploads
RAG is not the only way to use corporate knowledge with generative AI. RAG, fine-tuning and uploading documents into a long context window solve different problems.
| Approach | Primary purpose | Handling information updates | Evidence checking | Main design concerns |
|---|---|---|---|---|
| RAG | Retrieve relevant documents for each question and supply them as evidence | Can reflect changes through index updates | Retrieval results and citations can be shown | Retrieval quality, permissions, chunking, updates and auditability |
| Fine-tuning | Adjust output format, tone or behaviour for a specific task | Usually not the right tool for frequent knowledge updates | Not designed to identify the exact training passage supporting an answer | Training-data quality, evaluation and model operations |
| Long-context upload | Ask questions about a limited set of documents supplied for a session | Documents can be supplied again for each session | Evidence can be checked within the uploaded material | Input limits, repeated preparation, permissions and latency |
RAG is a strong candidate when a changing document collection must be searched repeatedly and users need answers with sources. Fine-tuning is not a universal method for making a model “remember” internal documents; it is better considered when the requirement is a particular answer format or task behaviour. If the need is a one-off analysis of a few fixed documents, a simple upload may be sufficient.
Start with the business requirement, not the technology label. Break the requirement into these questions:
- How frequently does the information change?
- Must users verify the source behind an answer?
- Is access control required by document and by user?
- Is this a one-time analysis or an operational system used continuously?
- Must the system say “I don’t know” when approved evidence is unavailable?
The more important freshness, citations, granular permissions and ongoing operation are, the stronger the case for RAG. Regardless of the approach, incorrect answers remain possible. RAG does not eliminate hallucinations. It provides relevant evidence at answer time and makes systematic evaluation and auditing more practical.
Four principles that determine RAG development success
1. Limit retrieval to one business process and one document collection
Adding documents expands possible coverage, but it also increases duplicate content, superseded versions, permission differences and inconsistent terminology. For a PoC, prioritise a collection that is used regularly, has objectively verifiable answers, has an accountable document owner and has a clear permission boundary.
Do not define scope by folder name alone. Create a document register containing, at minimum, document ID, title, revision, effective date, expiry date, owning department, confidentiality class, permitted groups, source URL, update frequency and index-update timestamp. Without this register, it is difficult to distinguish an outdated answer from a retrieval failure or an error in the source itself.
PDFs, spreadsheets, scanned images and slide decks also require different parsing treatment even though users see them all as “documents.” Confirm how the system preserves table-cell relationships, heading hierarchy, notes that cross page boundaries and text embedded in images. The improvements to layout parsing and chunking discussed by Google Cloud are particularly important for enterprise documents.
2. Preserve source-document permissions at query time
Ingesting a document into RAG does not automatically preserve the permissions of the source system. The index must carry access-control metadata, query-time filters must use a verified user identity, and only documents permitted for that user may be passed to the generative model.
Microsoft Azure AI Search describes design elements including document-level security trimming, Microsoft Entra ID permission metadata, query-time filters and private endpoints. Amazon Bedrock Managed Knowledge Base also supports document-level permission filtering, but AWS explicitly states that ACL-aware filtering is not, by itself, authorisation. The application must authenticate the end user and supply a verified identity context.
An RFP therefore needs more than a checkbox saying “supports document ACLs.” The acceptance test should verify this complete chain:
- Authenticate the user through the company’s identity platform.
- Obtain the authenticated identity and group memberships.
- Pass verified permission conditions with the retrieval request.
- Filter the document index using access-control metadata.
- Pass only authorised retrieval results to the generative model.
- Log the questioner, filter conditions, referenced documents and answer in an auditable form.
Do not pass the test merely because no confidential information appeared in the final response. A prohibited document might have been retrieved but happened not to appear in the generated wording. Inspect the retrieval logs and require zero prohibited documents in the result set.
3. Evaluate retrieval separately from the answer
When an answer is wrong, there are at least two possible causes: the system failed to retrieve the correct evidence, or it retrieved the correct evidence but interpreted it incorrectly during generation. If these are not measured separately, the team cannot tell whether to improve the prompt, the index or the chunking strategy.
Microsoft’s Azure Architecture Center recommends separating retrieval evaluation from end-to-end evaluation and assessing dimensions such as groundedness, completeness, utilisation and relevance. It also recommends recording hyperparameters and evaluation results and aggregating results across multiple questions.
For each question, an enterprise evaluation register should capture the following:
| Evaluation layer | What to verify | Example record |
|---|---|---|
| Retrieval | Did the correct document and passage appear among the candidates? | Expected document ID, retrieved IDs, rank and filter conditions |
| Evidence use | Did the answer use the retrieved passage appropriately? | Citations used and important passages omitted |
| Final answer | Did it answer the question correctly and sufficiently? | Accuracy, completeness, relevance and adjudication rationale |
| Safety | Did it avoid unauthorised or confidential information? | User role, prohibited documents and whether any leakage occurred |
| Operations | Did it refuse safely and provide guidance when evidence was insufficient? | Reason for refusal, confirmation point and escalation route |
Run the same question set while varying chunk size, overlap, search method, number of retrieved results and ranking configuration. OpenAI’s Vector Store Search API supports filters based on file attributes, query rewriting, ranking options and a maximum of 1 to 50 results. Availability of a setting does not mean its maximum value is best. Use the organisation’s own evaluation set to compare quality, latency and cost.
4. Build updates, audits and safe refusal into production operations
A production RAG system is not complete after the first ingestion. Microsoft’s advanced RAG guidance treats pre- and post-processing, chunking, update strategy, feedback, tracing and logging as production-system components. It also describes an assessment pipeline for checking whether the correct chunks were retrieved and whether they were divided appropriately.
The operating model should be able to answer these questions:
- When a source is updated, expired or deleted, when will the index reflect the change?
- Who detects a synchronisation failure and triggers a retry?
- Can the team trace the source revision and indexing time used for an answer?
- How do low user ratings feed into the evaluation set and improvement backlog?
- Can the system avoid guessing and refuse when evidence is insufficient?
- Can logs be preserved and the impact assessed after an incident?
A refusal is not necessarily a failure. In an enterprise setting, “This cannot be confirmed from the available approved documents; please contact the responsible department” is safer and more useful than a plausible answer without approved evidence. Acceptance criteria should measure not only answer rate but also the rate at which questions that should not be answered are refused correctly.
Choosing classic RAG, agentic RAG and an operating model
Treat retrieval design and operating responsibility as separate decision axes. Classic versus agentic RAG describes retrieval behaviour; managed versus customer-managed describes responsibility for the platform.
| Option | Mechanism and characteristics | Suitable requirements | Points to examine |
|---|---|---|---|
| Classic RAG | Typically retrieves candidates through hybrid search and semantic ranking | Straightforward questions, lower latency, understandable architecture and general-availability requirements | Questions with multiple issues may require query transformation or extra processing |
| Agentic RAG | Decomposes a complex question into subqueries, retrieves in parallel and combines results | Questions spanning several documents or issues | More execution paths, latency, evaluation dimensions and logging scope |
| Managed | Uses cloud capabilities for connectors, ingestion, parsing and indexing | Lower operational burden and use of standard platform features | Supported connectors, permission propagation, region and platform constraints |
| Customer-managed | The organisation operates the vector store, ingestion, parsing and indexing | Special data processing, tighter control or detailed tuning | Greater flexibility also brings responsibility for monitoring, updates and incident response |
Microsoft recommends agentic retrieval for new implementations, while also explaining that classic RAG may be appropriate where general availability, simplicity or speed is the priority. “Newer” should not be treated as automatically superior; choose based on question complexity and acceptance criteria.
For operating models, Amazon Bedrock’s managed Knowledge Bases can provide connectors for S3, SharePoint, Confluence, Google Drive and OneDrive, together with document-level permission filtering and multimodal parsing. A customer-managed Knowledge Base instead allows the organisation to operate the vector store, ingestion, parsing and indexing. OpenAI also provides an official Knowledge Retrieval blueprint combining File Search, Agent or Chat SDKs and Evals. These primary sources help teams understand candidate technologies; product names alone do not establish fit or superiority for a particular organisation.
Compare more than retrieval accuracy:
- How completely can the platform inherit the existing identity system and document ACLs?
- Within what time window can it reflect updates, deletion and expiry?
- How will it retrieve across Thai, Japanese and English questions and documents?
- Can it trace retrieval logs, generation logs, citations, model versions and configuration versions?
- Does it satisfy data-residency, network and secret-management requirements?
- Can the organisation migrate its document register, evaluation set and logs if the service changes?
A 90-day PoC plan for staged enterprise RAG decisions
Ninety days is a planning framework for separating discovery, development, evaluation and handover—not a guaranteed industry standard. Adjust the period to the condition of the documents and the organisation’s approval process. What matters is defining exit criteria at the end of each phase so the team can continue, revise or stop deliberately.
Days 0–15: Fix the business scope and evaluation set
During the first 15 days, interview the user department, define the business process, prepare the document register, map permissions and create the evaluation questions. Include common questions as well as ambiguous questions, questions requiring several documents, questions for which no answer exists, attempts to elicit restricted documents and malicious instructions.
Exit criteria for this phase are:
- The target process, users, document collection and exclusions can be described in one sentence.
- Owners of the authoritative documents and their updates are confirmed.
- Confidentiality classifications and permitted groups are known for every document.
- Evaluation questions with expected evidence and answers have been reviewed.
- Data that must not enter the PoC is explicitly documented.
If these conditions are not met, narrow the scope rather than rushing into development. A visually impressive demo cannot reveal quality regression after launch if there is no reference answer set.
Days 16–35: Build the minimum secure ingestion and retrieval flow
Implement the minimum configuration for document parsing, metadata assignment, chunking, indexing, authentication and query-time filters. At this stage, prioritise traceability from source to retrieval over interface polish.
Confirm that every chunk can be traced back to its source URL; revision, effective date, owning department and ACL are retained; deleted documents disappear from search; and prohibited documents never appear in unauthorised users’ retrieval results.
Exit criteria should include: authorised documents are retrievable and prohibited documents yield zero results for several authenticated roles; update and deletion test results match the document register; and ingestion failures can be detected. Do not advance if the permission test fails, even if the generated answers look excellent.
Days 36–55: Improve retrieval and answer quality separately
Execute the full evaluation set and score retrieval success, evidence use and final-answer quality in separate columns. For questions where the correct document was not retrieved, classify the cause among missing data, parsing, metadata, chunking, query transformation, search or ranking. Where the correct evidence was retrieved but the answer is poor, review instructions, context construction, citations, answer format and refusal conditions.
Every configuration change should record its settings, execution time, data version, question-set version, model and retrieval configuration, and evaluation result. Do not decide from one successful example; review aggregate results and failure patterns across many questions.
Exit criteria are that the agreed measures meet the thresholds established at the start of the PoC, permission tests continue to pass, and serious failures are classified reproducibly. The purchasing organisation must set its own thresholds based on business risk and the evaluation set. An unsupported universal benchmark is not useful.
Days 56–75: Add the user workflow, audit trail, refusal and feedback
Create an interface through which users can open citations, confirm the source revision, rate the answer and escalate to the responsible department. Logs should retain, to the extent necessary, the user, time, filter conditions, retrieved documents, answer, citations and configuration version. Because logs may themselves contain confidential information, define access permissions and retention periods for them.
Use the OWASP Top 10 for LLM and GenAI as a reference when adding tests for prompt injection, sensitive information disclosure, data or model poisoning, and vector and embedding weaknesses. Confirm that external documents and instructions embedded within documents are not trusted indiscriminately, suspicious retrieved instructions cannot override higher-level controls, and questions designed to extract confidential information are rejected.
Exit criteria are that major safety tests pass, refusal and escalation work, and an auditor can trace an answer back to its source.
Days 76–90: Acceptance testing, handover and the production decision
In the final phase, the user department, document owners, information security team, IT operations and, where appropriate, legal or personal-data specialists conduct acceptance testing. Test not only the evaluation set but also updates, deletion, permission changes, synchronisation failures, service failures, log review and support processes.
Production exit criteria should be explicit:
- Retrieval and answer evaluations for the target business process meet the agreed thresholds.
- Unauthorised documents never enter retrieval results or generation context.
- Updates, deletion and permission changes are reflected within the agreed operating window.
- Refusal, citation, feedback and escalation functions work.
- Owners are assigned for monitoring, incident response, re-indexing and evaluation reruns.
- Residual risks and user guidance are approved.
If the criteria are not met, choose among continuing with a restricted scope, revising the design or stopping. The purpose of a PoC is not to stage a success; it is to produce reliable evidence for an investment decision.
Estimate RAG implementation cost with a cost register, not a product price
A single licence or model price cannot represent total RAG implementation cost. Data preparation, permission integration, evaluation, monitoring and change management support the quality and safety of retrieval and generation. Avoid presenting an unverified market price. Instead, record the quantity, unit cost, frequency and allocation of responsibility for each requirement.
| Cost category | Items to include in the register | Main quantity drivers |
|---|---|---|
| Discovery / data inventory | Process analysis, document inventory, authoritative-source, owner and confidentiality review | Number of departments and collections, permission complexity |
| Connector and ingestion | Connection, extraction, parsing, OCR, metadata and synchronisation | Number of sources, formats, update frequency and failure handling |
| Retrieval / index | Indexing, vectorisation, hybrid search and ranking | Document volume, update volume, query volume and retention design |
| Model inference | Query understanding, response generation and required pre/post-processing | Question count, input/output volume and execution path |
| Identity / ACL | Authentication, group synchronisation, query-time filters and permission testing | Identity platforms, role count and ACL granularity |
| Evaluation set | Questions, expected evidence, scoring and regression testing | Business processes, languages, risk classes and revision frequency |
| Monitoring / logging | Tracing, monitoring, alerting, retention and audit | Usage, retention period and investigation requirements |
| Operations / change management | Document ownership, training, support, improvement and re-evaluation | User departments, change frequency and support model |
Separate initial and recurring costs. Initial expenditure typically includes inventory, design, integration, the evaluation set and acceptance testing. Monthly expenditure can include cloud consumption, synchronisation, monitoring, support, document updates, evaluation reruns and improvement work. Add separate columns for supplier work and internal work; this makes omissions easier to find when comparing quotations.
Hypothetical ROI example—not a market benchmark or outcome guarantee
The following assumptions illustrate the calculation method only. They are not results from a real company, market-standard prices or a guarantee of benefits.
- Monthly enquiries: 20,000
- Current information-search time: 6 minutes per enquiry
- Information-search time after using RAG: 2 minutes per enquiry
- Relevant labour cost: THB 450 per hour
- Share of RAG answers that can be adopted in the business process: 70%
The estimated monthly time reduction is:
20,000 enquiries × (6 minutes − 2 minutes) ÷ 60 minutes × 70% = 933.3 hours per month
Using the unrounded value, the monetary conversion is:
20,000 × 4 ÷ 60 × 0.70 × THB 450 = THB 420,000 per month
This THB 420,000 is a hypothetical efficiency opportunity, not automatically a profit increase or budget reduction. The decision must also consider question difficulty, answer-adoption rate, verification work, exceptions, operating cost and quality-incident risk. During the PoC, measure search time through usage logs or work observation, record answer adoption through user decisions, and replace every assumption with the organisation’s own evidence.
The ROI register should also cover quality. Incorrect source documents, use of superseded revisions, permission breaches and missed escalations cannot be evaluated through time saving alone. Depending on the process, the primary value may be easier access to approved evidence or auditability, rather than faster answers.
Enterprise RAG RFP checklist
An RFP should not say only “build a RAG chatbot.” It should communicate the business requirements, data, permissions, evaluation method, operating model and acceptance conditions. When every bidder responds against the same assumptions, the organisation can compare responsibility and delivery method as well as price.
Business and scope
- Target users, target process and role in decision-making
- Included and excluded documents, authoritative sources and owning departments
- Expected questions, multi-document questions and questions that should be refused
- Supported languages and combinations of question and document languages
- Boundaries among PoC, limited production and enterprise-wide deployment
Data and retrieval
- Supported sources, file formats, tables, images and scanned documents
- How document ID, revision, effective date, expiry date and source URL are retained
- Chunking and layout-parsing policy
- Options for hybrid search, ranking and query transformation
- Update, deletion, re-indexing and synchronisation-failure handling
- How retrieval results and cited sources are traced
Identity, permissions and network
- End-user authentication and enterprise identity integration
- Synchronisation of document ACLs and group information
- How verified identity context reaches retrieval
- Query-time filters and the data passed to the generative model
- Network isolation requirements such as private endpoints
- Separation of administrator, developer and auditor permissions
Evaluation and safety
- Method for separating retrieval and end-to-end evaluation
- Support for creating reference questions, expected documents and expected answers
- Treatment of groundedness, completeness, utilisation and relevance
- Tests for prompt injection, sensitive-information disclosure, data poisoning, and vector or embedding weaknesses
- Acceptance conditions for refusal, citations and escalation
- Regression frequency and approval after configuration changes
Operations and contract
- Monitoring, logging, alerts, retention period and audit method
- Allocation of communication and investigation duties after failures, incorrect answers or permission incidents
- Owners for document changes, model or retrieval changes and re-evaluation
- Initial costs, monthly costs, usage-based items and internal work
- Treatment of data, evaluation sets, logs and configurations at contract termination
- Handover documentation, operating procedures and training deliverables
Require proposals to describe test methods, not only feature availability: where the user is authenticated, which log proves that prohibited documents were not retrieved, and how deletion from the source is proven to have propagated to the index.
Specific items for RAG acceptance testing
A RAG demo can look excellent if it uses carefully selected questions. Acceptance testing must give equal weight to normal and failure scenarios.
| Test class | Input or action | Pass condition |
|---|---|---|
| Normal retrieval | A question whose answer is in one document | The expected document and passage are retrieved and the answer cites the source |
| Multiple documents | A question requiring several sources | The answer distinguishes the evidence and identifies contradictions |
| Refusal | A question with no answer in the target corpus | The system does not guess and states that the answer is unknown or gives a confirmation route |
| Version control | A question for which old and current revisions exist | The current revision is prioritised and its revision and effective date can be checked |
| Permissions | The same question from different user roles | Only permitted documents are retrieved; prohibited documents yield zero results |
| Permission change | Change a user’s group | Retrieval reflects the change within the agreed time |
| Document deletion | Delete or expire the source | The document leaves the retrieval scope within the agreed time |
| Adversarial input | A question or document containing prompt injection | Higher-level instructions, permissions and confidentiality remain intact |
| Logging | Audit any answer | The question, filters, retrieved documents, response and configuration version are traceable |
| Failure | Cause part of synchronisation, retrieval or generation to fail | The failure is not reported as success and can be detected, notified and retried |
Evaluators should inspect retrieval results, filters, referenced chunks and configuration versions—not only the generated prose. This distinguishes an answer that happened to be correct from one that is correct reproducibly.
Practical security, Thailand PDPA and governance considerations
In enterprise RAG, data exists at the input, index, retrieval-result, generation-context, answer and log stages. Use a data-flow diagram to identify what each stage stores, who can access it, how long it is retained and where it is sent.
NIST AI 600-1 is a voluntary, cross-sector risk-management profile for generative AI. It can help an organisation identify risks and consider controls, but should not be presented as a legal obligation. The OWASP Top 10 for LLM and GenAI is a useful reference for organising technical tests concerning prompt injection, sensitive information disclosure, data and model poisoning, and vector and embedding weaknesses.
For Thailand’s PDPA, Section 37 of the cited unofficial English translation indicates that a data controller should implement appropriate security measures to prevent improper or unlawful loss, access, use, alteration or disclosure, and review those measures when necessary or when technology changes. The cited document is an unofficial English translation. This article is not legal advice. Confirm applicability, lawful basis, cross-border transfer, processor management, notification and retention requirements with the organisation’s personal-data team and qualified advisers.
At minimum, the RAG security review should examine:
- Personal data collected and its business necessity
- Storage locations for source files, extracted text, embeddings, logs and backups
- How service providers and subprocessors handle data
- Separation of administrator, developer, user and audit permissions
- Encryption, secret management and network controls
- Procedures for propagating retention, deletion, correction and permission changes into the index
- Incident detection, investigation, communication and prevention of recurrence
Do not conclude that the system is safe because it has an ACL filter, uses a private connection or produces logs. Test the full chain from authenticated identity to retrieval filter, the permissions of log readers, and derived data remaining after the source is deleted.
Designing multilingual RAG for Thai and Japanese content
In a Thailand-based enterprise, a Japanese question may need evidence from a Thai document while product names and part numbers remain in English. Translating the interface alone does not validate multilingual retrieval quality.
Build an evaluation matrix for question-language and document-language combinations. Test not only Japanese-to-Japanese and Thai-to-Thai retrieval, but also Japanese questions against Thai documents, Thai questions against Japanese documents and questions containing alphanumeric model numbers. Prepare expected documents and answers for each combination, then score retrieval and generation separately.
For Thai, test word boundaries, spelling variants, abbreviations and department-specific terminology. For Japanese, test kanji, kana and Latin-letter variants, full-width and half-width characters, product names and internal abbreviations. Terminology and synonym dictionaries should be reviewed by document owners, with records of who changed them and when.
Whether to translate before retrieval, use multilingual embeddings or search in several languages should be decided using the organisation’s own question and document set. If translation is used, acceptance tests should verify that proper nouns, numbers, units, negation and revision numbers are preserved. Even when the answer language follows the question language, show citations in the source language and clearly distinguish any translation from the original.
For more on the broader design of multilingual assistants, see our Guide to Multilingual AI Chatbot Implementation in Thailand. When defining PoC KPIs and benefits, our Guide to Measuring AI Implementation Impact for Companies in Thailand explains how to separate business outcomes from quality measures.
An operating model for generative AI with internal data
If IT alone owns RAG, it cannot determine whether a document is authoritative or whether a business answer is acceptable. If the user department alone owns it, authentication, logging and incident response may not be sustainable. Separate these responsibilities at a minimum:
| Role | Primary responsibilities |
|---|---|
| Business owner | Approve the target process, usage scope, KPIs and acceptance criteria |
| Document owner | Manage authoritative sources, revisions, expiry, confidentiality and terminology |
| Data / retrieval team | Operate ingestion, parsing, indexing, retrieval evaluation and update monitoring |
| Application / identity team | Operate authentication, permission integration, interface and escalation |
| AI evaluation team | Maintain question sets, answer evaluation, safety tests and regression tests |
| Security / personal-data team | Review risks, controls, logs and incident response |
| Operations team | Execute monitoring, support, incident handling, changes and re-evaluation |
Changes after launch are not limited to model updates. New documents, layout changes, connector updates, permission-model changes, chunk settings and ranking changes can all affect quality. Run the same evaluation set before and after changes, then inspect retrieval and answer differences. Recording configurations and results helps identify whether an improvement for one question group degrades another.
User feedback should also be more specific than “helpful” or “not helpful.” Useful categories include: required document not found; citation correct but answer incomplete; superseded revision used; prohibited content visible; question intent misunderstood; and out of scope. Define how feedback is converted into new evaluation questions.
Common enterprise RAG failures and how to avoid them
Uploading a large document collection before defining the use case
As scope grows, so do duplication, obsolete revisions, permission differences and parsing failures. Define business questions and reference documents first, then ingest what is necessary. Expand enterprise-wide only after permissions, evaluation and updates work in a limited process.
Evaluating a PoC only by the appearance of its answers
Fluent language does not prove correctness. Record separately whether the expected document was retrieved, whether the citation supports the answer, whether the required content is complete and whether prohibited information was excluded.
Treating an ACL filter as authentication
ACL metadata cannot apply the correct permissions if the questioner’s identity has not been verified. Authenticate in the application, pass verified identity context into retrieval and test that prohibited documents never enter results.
Comparing only the initial RAG implementation cost
In addition to ingestion, indexing and model usage, the organisation must maintain evaluation sets, monitoring, permission synchronisation, document updates, support and change management. Compare initial versus monthly, internal versus supplier, and fixed versus usage-based items in a structured register.
Assuming RAG eliminates incorrect answers
Retrieval gaps, outdated sources, parsing errors and misinterpretation of evidence remain possible. Design for detection and impact reduction through citations, safe refusal, auditability, evaluation and escalation.
FAQ: Enterprise RAG implementation decisions
Which business process should an enterprise RAG implementation start with?
Start with one process and one document collection that are used regularly, have identifiable correct sources, have accountable document owners and have a clear permission boundary. Avoid an enterprise-wide first step. Validate retrieval, permissions, updates and refusal within a scope for which a reliable evaluation set can be created.
How should we choose between RAG development and fine-tuning?
RAG is a candidate when changing internal documents must be searched for each question and used with citations. Fine-tuning is more relevant when adjusting output format or task behaviour. A long-context upload may be enough for a one-off analysis of a few documents. If several approaches are combined, evaluate the requirement each one is intended to solve.
How should RAG implementation cost be estimated?
Break the estimate into discovery and data inventory, connector and ingestion, retrieval and indexing, model inference, identity and ACL, evaluation set, monitoring and logging, and operations and change management. Record quantities, unit prices, frequency and the division between internal and supplier responsibilities. Compare initial and monthly costs using the organisation’s document volume, usage, permissions and evaluation requirements—not an unverified market average.
What is the most important security control when using generative AI with internal data?
No single feature determines security. Design the complete chain: end-user authentication, verified identity context, query-time ACL filters, the scope passed to the model, audit logs, and propagation of updates and deletions. Test not only that prohibited information is absent from the final answer, but that prohibited documents are absent from retrieval results.
Can AI document search eliminate wrong answers?
No. Even with RAG, the system can fail to retrieve the correct document, use an outdated revision or misinterpret retrieved evidence. Manage the risk with separate retrieval and answer evaluation, citations, safe refusal, audit logs, regression testing and human escalation.
Should we choose classic RAG or agentic RAG?
Evaluate classic RAG where straightforward questions, latency, understandable architecture or general availability are priorities. Agentic RAG is a candidate for complex questions spanning several issues and documents because it can decompose queries and retrieve in parallel. Compare both using the same question set for retrieval quality, answer quality, latency and traceability rather than choosing by name.
Can RAG work when Thai and Japanese are mixed?
Yes, but it requires language-specific evaluation. Separate question-to-document language combinations and test variants, proper nouns, model numbers, numbers, negation and revision identifiers. Compare translation, multilingual embeddings and multi-language retrieval using the organisation’s own questions and documents.
Conclusion
Enterprise RAG implementation is not simply a choice of vector database or generative model. Its success depends on limiting retrieval by business process, preserving source-document permissions at query time, evaluating retrieval separately from answers, and embedding updates, audits and refusal into operations.
Begin with one process and one document collection with a clear permission boundary. Build an evaluation set and document register. During a 90-day PoC, use exit criteria to assess ingestion, permissions, retrieval, generation, safety, updates and auditability in stages. Estimate cost through a register covering data inventory, integration, ACLs, evaluation, monitoring and operations. Make the production decision from reproducible acceptance tests, not from an attractive demonstration.
If your RAG scope or RFP is still being defined, you are welcome to contact TOMAS TECH. We can help organisations in Thailand structure document, permission and multilingual requirements and identify the first business process worth validating—before a large implementation commitment is made.
References
- Microsoft, “RAG and Generative AI in Azure AI Search”
https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview
- Microsoft Azure Architecture Center, “Design and Develop a RAG Solution”
- Microsoft, “Build advanced RAG systems”
https://learn.microsoft.com/en-us/azure/developer/ai/advanced-retrieval-augmented-generation
- Amazon Web Services, “Amazon Bedrock Knowledge Bases overview”
https://docs.aws.amazon.com/en_en/bedrock/latest/userguide/knowledge-base.html
- Amazon Web Services, “Document-level access controls”
https://docs.aws.amazon.com/bedrock/latest/userguide/kb-managed-ds-custom-acl.html
- OpenAI, “Vector Store Search API”
https://developers.openai.com/api/reference/python/resources/vector_stores/methods/search
- OpenAI, “Knowledge Retrieval blueprint”
https://openai.com/solutions/blueprints/knowledge-retrieval/
- Google Cloud, “What is RAG?”
https://cloud.google.com/use-cases/retrieval-augmented-generation
- NIST, “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile”
- OWASP, “Top 10 for LLM and GenAI”
https://genai.owasp.org/initiatives/top-10-for-llm-and-genai/
- Thailand PDPA, unofficial English translation (Section 37 referenced)
https://pdpa.dmh.go.th/news/files/pdpa.pdf
*This article provides information for technical and operational consideration and does not constitute legal advice. Confirm the application of law with your organisation’s responsible personnel and qualified advisers.*
