When a factory introduces document version control into a RAG system, retrieval accuracy alone is not enough. The system may confidently cite a superseded work instruction and return a torque value, inspection frequency or maintenance sequence that is no longer authorized. In a Thai factory, the Japanese source, Thai shop-floor instruction and English customer document may be approved on different dates. This guide explains how to expose only approved and effective revisions, synchronize changes, retire old chunks, update permissions, and specify FAT/SAT and RFP acceptance criteria.
The real risk is a plausible answer from an obsolete revision
Imagine that the tightening instruction for machine M-04 changes from revision 2 to revision 3. Quality assurance approves revision 3 and production engineering distributes the Thai instruction, but text extracted from the revision 2 PDF remains in the search index. An AI answer may cite “revision 2, page 4” correctly. That citation proves the sentence came from a source; it does not prove that the source is valid for today’s work. Retrieval relevance and authorization for use are separate decisions.
The same problem applies to maintenance instructions, chemical handling, visual inspection criteria, customer-specific packing specifications and start-up checks. Search by semantic similarity can combine passages belonging to a different customer, line, site, machine or effective date. The risk becomes sharper when headquarters approves a Japanese original before the Thai translation is reviewed. A localized answer must not silently treat an old Thai version as equivalent to the newly approved Japanese revision.
ISO’s public material on document control calls for approval before issue, identification of the current revision and prevention of unintended use of obsolete documents. The RAG fields and tests below are our proposed implementation of that control logic, not a claim that ISO mandates a specific AI architecture or that RAG confers certification. See the ISO publication on document control and its guidance on documented information.
This article starts after the investment decision. For architecture, multilingual search and cost, see our factory RAG implementation guide. For PoC and return-on-investment decisions, see the enterprise RAG adoption guide. Here we focus on what happens whenever a controlled document changes.
Decide who declares a revision effective before choosing the search stack
Create a responsibility matrix before indexing PDFs. Quality assurance owns release approval and effective dates. Production engineering owns applicability to machines, lines and processes. Language owners approve the correspondence of translations to the source. IT owns synchronization, access controls and monitoring. A vendor’s statement that revision 3 was “successfully indexed” is not the same as quality assurance’s decision that revision 3 may be used on the shop floor.
A practical document register should hold a stable logical document ID, revision, state, effective time, site, line, equipment, language, source-revision relationship, approver, confidentiality class, access groups and authoritative location. A filename is a weak key: renaming or saving a PDF again can create a second indexed object while old chunks remain. Use an ID such as WI-M04-TORQUE independently of Rev. 3. Attach both to every extracted chunk so that updates and removals can be audited as a group.
At a minimum, distinguish draft, under review, approved but not yet effective, effective, suspended and obsolete. Approval and effectiveness are different. If revision 4 is approved today but starts after the night shift, revision 3 remains effective until then. If revision 3 is suspended urgently for a safety reason, the system may need to stop answering even when revision 4 does not yet exist. A simple latest=true flag cannot express all of these states; the retrieval gate should consider approval, effective time, applicability and suspension.
When the Japanese original and Thai instruction move at different speeds
Treat the source and translation as separate approval events. Record which Japanese revision each Thai revision translates, when the translation was approved, and where it applies. Selecting the highest revision number across languages can mislabel an old Thai instruction as current. Nor should the system convert a newly approved Japanese instruction into an authoritative Thai shop-floor procedure merely by machine translation. Agree on fallback behavior: for ordinary operators, “No approved Thai instruction is available; contact QA” may be the right answer. Authorized engineers may instead receive a clearly marked link to the Japanese source. Translation quality is secondary to release authority.
Make approval-to-search release a controlled change procedure
Do not publish a document to retrieval merely because a file changed. Separate candidate ingestion, text extraction, chunking, metadata validation, test retrieval, QA approval and live release. In a staging area, check broken links, tables, values embedded in images, units and differences from the previous revision. Only then should the document become eligible for operational answers.
The release unit is a logical document and all of its derived chunks, not one PDF file. If revision 3 produced eight chunks and revision 4 produces seven, allowing both sets in live retrieval can splice contradictory instructions together. Depending on the platform, one may switch searchable metadata, switch an index alias, or delete old chunks before enabling new ones. Verify whether the chosen platform makes the transition atomic. If it does not, pause answers for that document during transition rather than exposing a mixed state.
Rollback needs its own decision. If revision 4 is withdrawn, does revision 3 become effective again, or is the instruction suspended until QA decides? The factory, not the software, owns that rule. Define how the index, citations, ACL metadata and caches are restored, and log who approved the reversal and why. Past chat answers cannot always be erased; at least show the answer time and cited revision and prompt users to recheck before reusing old instructions.
Test changed, deleted and permission-modified documents separately
Incremental synchronization handles new and changed source documents where the connector and data source support change detection. Microsoft explains that Azure AI Search indexers can process only new or updated documents in such configurations. Deletion requires separate handling. With Azure Storage, a correctly configured soft-delete detection policy allows an indexer to remove corresponding search documents. Physically deleting the source too early can leave orphaned search content. This is connector-specific behavior, not a universal promise. See Microsoft’s indexer guide and change/deletion guidance.
AWS also states that Amazon Bedrock Knowledge Bases must be synchronized after additions, modifications or removals; a sync run incrementally processes these changes. Approval in the source repository therefore does not instantly change retrieval. The team needs a trigger or schedule, completion check, error notification and retry procedure. AWS provides an explicit deletion API for documents in applicable custom data sources, but the exact removal path depends on the connector. See the Bedrock synchronization guide.
A successful job status is not proof that an obsolete instruction is absent from answers. Inspect processed counts, failed files, retry queues and indexed records, then search for phrases and values unique to the old revision. Confirm both that the new revision can be retrieved and that the old revision cannot be used as a current work instruction. Audit retention may require keeping the old PDF in a records repository; it does not require keeping it in live operational retrieval.
Detect missed events with reconciliation
Nightly batches, manual uploads, outages or insufficient connector permissions can cause an event to be missed. Alongside event-driven updates, compare the register’s effective-revision list with the index’s searchable-revision list. Use logical ID, revision, language, scope and state as comparison keys. Flag old revisions present only in the index, newly effective revisions missing from it, unexpected chunk counts and stale ACL metadata.
Define the response to a discrepancy before production launch. For safety- or quality-critical work instructions, a prolonged sync failure should normally suspend answers for affected documents and direct users to the controlled source. Less critical HR or administrative material may permit a warning for a limited period. Risk classification keeps the factory from treating every mismatch identically while preserving a clear escalation path.
Permission changes belong in the same change pipeline
Quality investigations and customer drawings may be visible only to a department, project or customer team. Blacking out text after retrieval is weak: a forbidden title, snippet or citation may already have leaked. Authenticate the caller and filter retrieval to documents that user may access. The source link in a citation must pass the same permission check.
Azure AI Search describes security filters and document-level ACL/RBAC approaches, but some native ACL and SharePoint permission ingestion features are preview. In the SharePoint preview connector, the 2026-05-01-preview API and later can pick up ACL changes on items with unique permissions on the next successful indexer run. Changes inherited from a parent site, library or folder require an explicit permission resync or affected-document refresh. Ask the vendor for the exact API version, inheritance pattern and update procedure; do not accept a generic promise of automatic propagation. See document-level access control and security guidance.
For Amazon Bedrock custom data sources, ACL-aware retrieval depends on supplied ACL metadata and verified caller identity. AWS explicitly says ACL awareness is filtering, not end-user authentication. The application must authenticate the person and pass trusted identity context. Test what happens when an employee transfers departments, leaves a customer project or leaves the company. A document with unchanged text can still have a changed ACL, so “no content update” is not a reason to skip the permission update. See AWS document-level access controls.
For FAT, prepare accounts for production engineering, QA, an ordinary operator, an unrelated customer team and a former employee. Check that permitted documents appear, but give equal weight to denial: forbidden material must not show up in search results, summaries, citations or suggested questions. A contract can specify zero forbidden hits in the test set. This is an illustrative acceptance condition, not a vendor accuracy claim. At a shared Thai shop-floor terminal, also test logout, session expiry, caches and downloaded source files; search filtering alone does not control an abandoned session.
Build a stale-answer test set before commissioning
Select a representative set of current instructions and identify passages whose values, actions or applicability changed across revisions. An initial set of 10–20 documents can make a manageable pilot, but the actual size must follow document volume and risk. For each question, record the expected answer, required document ID and revision, forbidden old revision, applicable site and line, user role, and the behavior when the answer is not authorized.
For example: “What torque applies to M-04 on line B during the night shift?” must not return a value for line A. “What was the former inspection frequency?” may be answered for a user with history privileges, but it must be labelled as historical and must not appear as today’s work instruction. “The Japanese revision 4 is approved but the Thai revision is not; tell me the Thai procedure” must follow the agreed hold or limited-reference behavior.
Do not reduce acceptance to one average accuracy score. Nine correct questions about uncritical equipment names cannot compensate for one unsafe old torque value. Score current-revision citation, obsolete-version exclusion, applicability, ACL denial and abstention when the controlled source is missing separately. Keep the query, role, timestamp, retrieved ID/revision, generated answer, reviewer decision and retest result. On the answer screen, display document name, logical ID, revision, effective date, applicable site or line, cited passage and accessible source link. “Source: manual.pdf” is too little information for a shift supervisor to validate a work instruction.
FAT: demonstrate the complete revision path
During factory acceptance testing, reproduce changes with test documents before connecting to live lines. A polished chatbot demo is insufficient. The customer should provide scenarios that start in the document register and end at a query. Include identical filenames with different IDs, the same ID across revisions, a new revision with fewer chunks, an English version released before Thai, an approval withdrawal and a permission revocation.
Put separate rows on the acceptance sheet for: unapproved content excluded; an approved but future-dated revision excluded; only the new revision cited after its effective time; obsolete chunks absent; old-only values not delivered as current instructions; denied users unable to retrieve restricted material; and failures reported and answered according to the agreed hold rule. Have QA compare the generated answer with the controlled original as well as reviewing automation logs. Set timing and numeric thresholds for this factory in the RFP; do not borrow an unsupported universal “99% accurate” target.
Measure elapsed time from approval to searchable release, but do not accept on timing alone. Verify old-chunk removal, cache changes, ACL propagation and citation link integrity after the job finishes. Define who is alerted if the deadline is missed, which answers stop, who authorizes restart and where the incident record is stored.
A step-by-step acceptance scenario
Use a synthetic document called WI-M04-TORQUE. Begin with revision 2 effective on line B and a test condition called “Value A”; avoid real machine settings in the trial. Ask the same question as QA and as an operator and confirm that both cite revision 2. Then register revision 3 as a candidate with “Value B”. Before approval, neither the candidate nor Value B should appear in operational retrieval. After approval, set its effective time for the next morning. Record the answer immediately before and after that boundary to verify that the cited revision switches at the agreed time.
Remove one paragraph from revision 3 and ask using a distinctive term found only in revision 2. A correct new citation alone is insufficient: the removed paragraph must not return as a current work instruction. If the implementation deletes old chunks, inspect the remaining records by logical ID. If it hides old chunks through filters, test that ordinary users cannot retrieve them and that historical access has its own permission. Where caching is used, save both the first answer after release and repeated answers.
Next leave the Thai instruction at the equivalent of revision 2 while Japanese revision 3 is effective. The Thai operator should receive the agreed hold response; an authorized manager opening the Japanese source must not see an unapproved machine translation labelled as official. Revoke an operator’s access and check search results, source links and conversation summaries immediately, after the next synchronization, and after cache expiry. Finally suspend revision 3 urgently: the operational answer should change to a hold state, and QA approval should be required for reactivation.
Attach this scenario to the RFP and ask each bidder for operation timestamps, register values before and after, sync job IDs, indexed revisions, screenshots of answers and reviewer decisions. Have them demonstrate detection, alerting and answer suspension when the target propagation time is missed—not just quote a speed. Reuse the FAT sheet during SAT with the real site configuration to expose features that worked only in a demo environment.
SAT: prove the workflow at the Thai site
Site acceptance testing uses the real document management system, Thai identity service, shared terminals, network, shift pattern and authoritative repository. A connector that worked in FAT may fail because of site-specific paths, SharePoint permissions, scanned PDFs or unstable links. Reproduce the actual sequence from Japanese headquarters approval to Thai translation approval, including a release during a weekend or night shift.
Ask local supervisors to query in Thai and independently check applicability, revision and cited passage. Natural translation matters, but exact units, model numbers and refusal to present an unapproved translation as an official work step matter more. If originals are scans, test OCR on tables, footnotes and image-embedded values. Drawings and photographs may require a separate review path if text extraction does not capture the controlled change.
SAT should end only when local owners can register a revision, detect a failed sync, remove an obsolete revision from retrieval and activate the emergency hold themselves. Handover includes operating procedures, log location, escalation contacts, recovery practice and a clearly assigned owner. A system that requires a vendor ticket for every routine revision has not solved version-controlled operation.
Write change events and evidence into the RFP
“Always answer from the latest document” is too vague. A vendor may interpret “latest” as newest file timestamp, largest filename suffix or latest approved effective revision. Define the register, business authority and acceptance evidence first.
| RFP topic | Ask the vendor to specify | Acceptance evidence |
|---|---|---|
| Identification | Stable ID, revision, language, site, line and effective date | Register-to-index comparison |
| Release gate | Treatment of draft, approved future, suspended and obsolete states | State-specific queries |
| Synchronization | Add, change, delete triggers and retries | Job history and exception list |
| Old-version removal | Chunks, citations and cache invalidation | Search using old-only terms |
| Permissions | ACL source, propagation time and identity boundary | Allow/deny account tests |
| Translations | Response when Thai is not approved | Language-specific tests |
| Recovery | Withdrawal, hold, rollback and audit trail | Logs and recovery drill |
| Handover | Ownership of register, extracted text, test set and settings | Export and operator demonstration |
Separate one-time ingestion cost from ongoing monitoring, translation approvals, reindexing, exception handling and test-set maintenance. Before comparing software prices, find out whether the existing document management system exposes approval status, ACL and retirement events. If it only exposes file contents, a register integration or manual publication gate will be needed.
Attach a synthetic scenario to the RFP: a work instruction changes from revision 1 to 2, one numeric value disappears, Thai approval is delayed and a user’s permission is revoked. Ask every bidder to run the same input and pass the same criteria. Require them to separate what worked in the demo, what works in the proposed production setup and what depends on preview features.
Operations after go-live
After launch, QA remains the owner of the controlled revision, production engineering confirms process applicability, IT monitors sync and access, and supervisors observe how answers are used. A vendor can troubleshoot but should not become the business authority for document release. Review update delays, register/index mismatches, obsolete citations, legitimate abstentions and failed permission denials. Add each meaningful incident to the test set.
If users often ask for “the old version”, determine whether they legitimately need history for an investigation or cannot find the current instruction. Historical lookup and operational instruction may need separate interfaces and privileges. Label the former “historical—do not use for work” and the latter with its current applicability. Automate checks at each change, reconcile periodically, and review exceptions at an operational meeting. The appropriate cadence follows the change rate and risk of the documents.
FAQ
Can a filename containing “Rev.” provide factory RAG version control?
Usually not. The filename does not reliably express approval, effective date, translation relationship, line applicability, retirement or ACL changes. Maintain a stable document ID and revision metadata in a register, propagate them to all chunks, and test whether renames or resaves leave orphan content.
Does successful incremental sync eliminate obsolete answers?
No. Change detection and deletion detection are different, and old chunks, caches or citations may remain. Query old-only terms and values after every test update, then inspect failed files and pending retries. A green job status is one signal, not proof of correct answers.
What if the Japanese source is approved but Thai translation is pending?
The document owner should define the translation’s validity and fallback. For ordinary operators, a clear “No approved Thai instruction” response may be appropriate. Authorized specialists may view a labelled source document, but machine-translated text should not quietly become an approved work instruction.
What should a factory RAG FAT/SAT accept?
FAT should test pre-approval, future effectiveness, revision switch, old-version removal, retirement, ACL changes and sync failure. SAT repeats those flows with the site’s real identities, repository, language, terminals and shifts. Treat non-retrieval of obsolete and forbidden content as distinct pass conditions.
Can an existing document management system remain the source of truth?
Often yes, if the integration can obtain approval state, revision, effective date, ACL and retirement events as well as content. A connector that reads only text needs an additional register integration or a controlled export of approved documents. Prove deletion and permission updates with the actual connector.
Conclusion
To introduce factory RAG document version control, first assign revision authority and define how “current” is determined. Treat approval, effectiveness, retirement and permission changes as search change events. Design the old-to-new transition as one controlled release, then test stale answers, unapproved translations and denied access in FAT and SAT. Compare bidders on reconciliation, failure holds and handover evidence rather than a generic promise to “use the latest document”.
TOMAS TECH can help Thai factories define the document register, stale-answer tests and FAT/SAT checklist even before selecting a RAG product. If you are assessing a live document management system, share the target process through our contact page.
