When an incoming or in-process part fails inspection, the immediate task is to identify and hold the physical item, establish which specification revision applies, and assemble reliable evidence for an authorized disposition. AI can flag missing records and contradictions. It should not independently decide rework, repair, scrap, or concession, and it should never release inventory because a model assigned a high confidence score. This guide covers the workflow from identification through MRB (material review board) decision and ERP/MES/QMS release gates, then shows what to require in an RFP and test at FAT and SAT. The examples are proposed design patterns, not measured results from a particular factory.
Separate disposition from corrective action
Disposition answers what to do with this item or lot now. Corrective action investigates causes and prevents recurrence. They can refer to each other, but a pending 8D investigation cannot justify leaving a suspect lot unmarked or available for shipment. Likewise, finding a visually similar defect photograph may help search previous cases, but it does not establish the applicable specification, decision authority, or release conditions. See our separate articles on AI assessment of corrective-action effectiveness and defect-photo similarity search.
The present article focuses on evidence cross-checking before an authorized MRB decision, across all disposition paths. The execution details of rework control and the approval conditions for concession management are separate subjects. Rework generally aims to bring the product back to specified requirements; repair may make it fit for intended use without fully restoring those requirements. The exact definitions and permitted routes must come from the organization’s applicable standards, customer agreements, and product regulations.
A single “NG” status is too coarse. Suspected, confirmed, physically segregated, awaiting evidence, under MRB review, approved but not executed, awaiting reinspection, awaiting customer approval, and released are different states. If a warehouse can allocate material while quality is still reviewing it, the system has not controlled the nonconforming product. An AI layer cannot compensate for a missing hold gate.
Define a narrow first deployment
Start with nonconformities found at receiving and at one in-process inspection point. Incoming cases need purchase order, supplier, receipt lot, part number, receipt date, inspection result, applicable drawing and purchasing-specification revisions, and a link to the physical label. In-process cases need work order, operation, equipment, serial or lot, consumed materials, work time, work-instruction and inspection revisions, measurement instrument and calibration status, and measured values. Do not send all these data to a model by default; present only the evidence required for the decision while preserving links to authoritative originals.
Keep customer returns, RMA, warranty and complaint workflows outside the first scope. They have different logistics, accounting, and approval rules. Include final inspection only if the same identification and hold controls can be applied. Define whether control is at serial, lot, container, or batch level. A “release whole lot” button is unsafe when only selected serials have passed the required review; a serial-level workflow is unusable when physical material has only lot identity. Specify partial hold and partial release, as well as split and merge histories, in the RFP.
Identify and segregate before analysis
At the moment an inspector raises a suspected nonconformity, link the physical item, quantity and location to a case ID. Scan its barcode or QR code, place a hold label, and move it to a clearly marked quarantine area where practical. For heavy or in-machine work in progress, define an alternative physical control such as a locked location or unmistakable equipment-side indication. A digital hold without a corresponding shop-floor control leaves room for unintended use.
One possible system split is that QMS owns the nonconformity case, MES blocks progression to the next operation, and ERP or WMS blocks allocation and shipment. Existing installations may use another owner. The essential decision is which system owns the final “shippable” state and how that state reaches every executing system. A quality screen that says “released” while ERP remains held, or an ERP release while QMS still says “hold,” must trigger reconciliation rather than a silent assumption.
Release must depend on explicit conditions: disposition approval by the authorized role; customer approval where required; execution of the approved work; passing the specified reinspection; quantity and serial or lot reconciliation; and a traceable signature or approval event. “Approved but not yet reworked” and “reworked but not reinspected” remain held. An emergency override, if the organization permits one, needs its own authority, scope, expiry, rationale and review under customer requirements. It is not an AI score threshold.
What AI should cross-check
Use deterministic rules for exact matches and hard limits. They can compare purchase-order part number with inspection record, unit of measure, instrument calibration date, mandatory fields, and quantity totals. Use AI where unstructured specifications, drawings, notes or multilingual text require retrieval and relationship checking. Examples of useful flags are a drawing revision that conflicts with the manufacturing order, a measurement in the wrong unit, a photo without serial identity, or a specification paragraph that appears to require customer authorization.
The newest document is not automatically the applicable one. An older revision may validly apply to a lot because of the order date, effectivity date or customer change notice. The workflow should determine applicability using those facts; if it cannot, it should show “revision unresolved” for human review. The screen should display each finding beside the source document ID, revision, page or paragraph, and the unresolved evidence. OCR uncertainty, inaccessible documents and network retrieval errors must be separate from an actual “not required” result.
In particular, distinguish four meanings of “missing”: never entered, original exists but access is denied, temporary retrieval failure, and genuinely not applicable. The first three are not evidence that no issue exists. Keep the hold and permit re-check after recovery. A model confidence value cannot turn an inaccessible original into verified evidence.
Consider an illustrative incoming lot with an out-of-tolerance diameter. The AI cross-check first compares the PO, drawing revision, gauge ID, calibration record and unit. A revision conflict stops the evidence pack before anyone recommends a disposition. Once the revision is confirmed, the system may show whether other items in the lot have results and whether the feature is customer designated. The MRB member then examines the originals and records the decision. This is a design example, not a claim of measured deployment performance.
Consider a second illustrative case: a torque sensor briefly loses connectivity. Missing readings are “unavailable,” not “pass.” Hold the affected items, examine the equipment source log and identify the time and serial range. AI may organize surrounding records as context, but interpolated values are not measured inspection results. A qualified person decides whether retesting or another route is allowed. The model input, output, rule and prompt version, execution time, reviewer correction and reason should be retained in the audit trail.
Put MRB authority into the workflow
MRB is an authorized decision process involving quality and, as needed, engineering, manufacturing, purchasing or commercial functions. Define who may open a case, assess technical impact, approve rework instructions, approve a concession, request customer approval, and release the lot. Route approvals by safety, function, regulatory or customer-designated characteristics as well as financial value. AI can recommend the correct route under that authority table; it cannot borrow a user’s authority and approve in that person’s name.
Each disposition path needs distinct evidence. Rework needs an approved instruction, affected quantity, operator record and reinspection. Repair needs a documented remaining deviation and fitness-for-use assessment, plus customer approval where required. Scrap needs physical destruction or segregation linked to the ERP inventory adjustment. Concession needs defined deviation, scope, validity, approver and evidence of customer consent where required. These are implementation examples; the organization must decide mandatory fields against its QMS, contracts and regulations.
Model state transitions as Detected → Held → Evidence assembled → MRB review → Disposition approved → Work executed → Reinspection/verification → Release or scrap completion. Add branches for request for more evidence, customer response, partial quantity, decision change and repeated failure. Record changes as new events rather than overwriting prior approvals. For a lot of 100 items with 10 known failures, do not infer that the other 90 pass automatically; sampling rules and possible common cause may require broader hold. Track held, under-investigation and disposed quantities separately and reconcile them to the receipt quantity.
Make QMS, MES and ERP state consistent
The RFP should name the authoritative system for case ID, part and serial identity, quantity, hold reason, disposition code, approval, reinspection and release time. It should also state who may update each field and which systems receive events. A QMS case ID may be the common reference that links MES work order and ERP inventory line, but a plant with another ID architecture needs a maintained mapping and change history.
Give each event a unique ID and version. Replayed events must be idempotent; out-of-order older events must not overwrite a newer hold or release. If the release message to ERP fails, shipment remains blocked until delivery and reconciliation succeed. If ERP is released but QMS is held, raise an alert and stop the operational flow. Define and measure acceptable synchronization delay rather than promising “real time” without a testable number. Decide before deployment whether customer drawings may be sent to an external model; restrict or mask data based on confidentiality and separate access by customer, product and plant.
Agree on a pre-disposition decision matrix
The division of work needs to be more precise than “AI versus humans.” For each question, name the original evidence, an exact rule where one exists, the condition that makes the result indeterminate, and the person who resolves it. Part identity can be checked across PO, label and inspection record; if one is unreadable, send it back to receiving. A drawing revision may depend on an effectivity lot and a customer change notice, which calls for engineering or quality review. AI may identify a customer-designated characteristic in text, but the final classification should be checked against the controlled characteristic list.
| Question before disposition | Primary evidence | System role | If unclear or conflicting |
|---|---|---|---|
| Which physical item? | Label and receipt/manufacturing record | Compare part, lot and quantity | Hold and verify identity |
| Which specification applies? | PO, change notice, drawing, inspection standard | Compare revision and effectivity | Do not guess; return to engineering/quality |
| Are readings credible? | Raw data, instrument and calibration record | Check unit, time and calibration | Review whether remeasurement is possible |
| Who may decide? | Authority table and customer contract | Display required approvers | Reject unauthorized approval |
| Is release allowed? | Decision, work, reinspection, customer consent | Find missing conditions | Keep hold if any condition is open |
This is an RFP design aid, not a mandated legal form. An exceptional case should be registered for authorized review and then reflected in controlled rules and test data. A free-text “release anyway” shortcut can conceal a repeatable gap.
Establish baseline data and success measures
A useful pilot takes closed cases from one incoming part family and one process inspection point, including revision changes, partial holds, unit differences, failed reinspection, customer approval, missing originals and multiple cases against one lot. There is no universal minimum case count: choose a set large enough to represent the plant’s actual conditions and record why it is representative. Check whether part numbers, revisions and photographs can be joined to authoritative records. OCR and entity matching can help, but false joins need a detection and correction path.
Measure the proportion of missing or contradictory evidence found, false-alert rate, time to a complete MRB packet, number of returned packets, misuse or shipment of held items, time to locate an approval trail, and time to resolve state mismatch. State the denominator for each metric and compare like-for-like scope before and after. Do not equate a model’s disposition predictions with business value. Calculate savings from the plant’s own measured baseline; rare but severe mis-shipment risk should be assessed separately from routine time savings.
Mandatory RFP requirements
An RFP should separate physical control, revision-aware evidence, approval authority, system integration and acceptance tests. Ask vendors to demonstrate the failure paths with realistic sample data, rather than only a polished screen for a complete case.
| Area | RFP requirement | Acceptance evidence |
|---|---|---|
| Physical identity | Link part, lot, serial, quantity, location and hold label | Scan matches the item and case |
| Revision | Resolve applicable specification with effectivity rules; show ambiguity | Mixed-revision test case |
| AI cross-check | Show missing and conflicting evidence with original source; support “cannot determine” | Reproducible source and log |
| MRB authority | Role separation and customer-approval hold | Unauthorized attempt denied |
| Execution | Work record and passing reinspection are release conditions | Incomplete work remains held |
| Integration | Owners, delay, retry, duplicate and outage behavior | Fault-injection record |
| Audit | Preserve document, model, prompt, input, output, correction and approval versions | Searchable audit export |
For a Japanese-owned plant in Thailand, include Japanese and Thai field labels, work instructions and return reasons if both languages are used on site. Keep the disposition code itself common across languages. Compare vendors by whether they fail safely on missing evidence, detect inconsistent states and explain the result with the original record, not by counting AI features.
Use FAT and SAT to test blocked paths
At FAT, test a lot with two cases, a valid older revision, an unavailable source document, an unauthorized approver, a concession without customer approval, duplicate messages, a completed rework without reinspection, and an out-of-order event. All unsafe cases must remain held. Also prove the positive path releases only the exact approved quantity. Evaluate AI alerts against independently labeled cases, reporting missed issues and false alarms separately.
At SAT, use actual scanners, labels, shelving, user roles, network and QMS/MES/ERP links. Test wrong-lot scans, a disconnected workstation, a delayed ERP message, an unavailable night-shift approver and a pick attempt from quarantine. If paper fallback exists, test recovery and duplicate-release prevention. Record inputs, expected result, actual result, screenshots or event IDs, defects, retest result and sign-off. IEC 62381:2024 specifies FAT/SAT test planning for automation systems in the process industry; it is a useful reference for agreeing scope and responsibilities, not a blanket requirement for every discrete-manufacturing plant.
Five failure paths to test
First, retrieval succeeds but returns a similar-looking revision for another customer; matching document names are insufficient without customer, part and effectivity. Second, OCR reads a tolerance sign incorrectly; show the original image next to structured critical values. Third, hold and release events arrive in reverse order after a network outage; an older release must not supersede a newer hold. Fourth, a former approver’s old session is reused after their authority changes; preserve history while requiring a valid current approval. Fifth, only part of a lot passes reinspection; release only the identified quantity if contracts and procedure permit partial release, otherwise hold the lot.
When a test fails, investigate document indexing, revision rules, permission-cache lifetime, event ordering or quantity model, rather than only rewriting a screen message. Re-test the failed case and adjacent legitimate cases. A fix that rejects old revisions must still accept a lot for which the old revision is valid. Link FAT and SAT evidence to requirement and case IDs so a future design change identifies affected tests.
The AI component also needs monitoring after deployment. NIST’s AI RMF discusses documented roles and responsibilities, testing before deployment, and periodic evaluation in operation. A changed document format, a new product family or increasing false alerts should trigger impact review and re-test before model, threshold or prompt changes are promoted. Keep those changes versioned.
Deploy in stages and train the floor
Stage one establishes physical identity, hold, MRB approval and release gate without AI. Stage two adds revision and evidence cross-checks. Stage three can add prior-case retrieval and natural-language evidence organization. In every stage, reviewers must be able to open the original source and correct an AI flag with a reason. For a Thailand plant using Japanese QMS rules, Thai shop-floor instructions and English customer documents, approve the translated terms in advance while keeping part IDs, disposition codes, authority and revision numbers common across languages.
Training should cover work that must not be performed while an item is held, correction of a wrong AI flag, and the contact path during a network outage, not just button clicks. Ensure “concession” and “customer approved” carry the same controlled meaning on every language screen. If rollout later includes a Vietnam site, treat local display and authority differences explicitly while keeping common identifiers.
Do not use age of held inventory alone to pressure quick release. Categorize delay as evidence, engineering, customer, execution or interface waiting. A warning that operators always dismiss calls for analysis of model accuracy, rules, presentation and training. Feed verified false alarms back into the test set rather than hiding the warning.
Govern changes after go-live
A drawing revision will not automatically update every cross-check correctly. Assign owners for change notice receipt, effectivity-lot entry, review of in-process items under the old revision, and update of AI retrieval. Track both old and new revisions until the transition is reconciled. The change record should list affected products and open cases, before/after revisions, effectivity, tests and approvals. A prompt update alone may make the explanation on screen inconsistent with the MES inspection standard.
At a locally defined review interval, quality should examine true and false alerts, missed evidence gaps discovered by people, overturned AI flags and cases placed back on hold. Alert count alone says little when product mix or document format changes. Classify causes as source revision, OCR, retrieval, authority, interface or training. For an audit, take a sampled shipment and trace backward through ERP inventory, MES operation, QMS case and MRB approval, measured values, and the applicable specification revision. A polished AI summary is not traceability when a link in that chain is broken. NIST’s manufacturing data guidance concerns data provenance and trustworthiness; it does not authorize a particular factory’s MRB decision.
Document responsibilities across quality, the shop floor and IT. Inspectors own physical identity and source measurements; quality checks evidence and segregation; engineering reviews applicability and impact; the authorized MRB member decides disposition. IT repairs failed integration but has no authority to remove a quality hold. Define limited substitute approvals for nights and holidays in advance or keep the hold. If an operator selected the wrong lot, correct it through an auditable event and examine whether other items were affected. If AI cited the wrong document, repair the index and search other open cases for the same false join.
During a paper-to-digital transition, name the formal record and reconciliation point. Compare paper labels with electronic holds at shift handover; stop shipment on mismatch. Retain a controlled outage procedure and reconcile on recovery. Deploying an AI screen without changing warehouse pick controls or physical labels does not secure the overall process.
Define measurement periods and denominators in the benefit report. “Time to MRB” may end when an evidence packet is ready or when a decision is signed; those are different measures. Separate elapsed time from working hours. For evidence-gap detection, count independently verified gap cases as the denominator and correctly flagged cases as the numerator; report the case counts alongside percentages, especially when the sample is small. Zero mis-shipments alone does not prove the control worked. Check rejected pick attempts and release-gate audit records. Put metric definitions in the RFP and test plan before deployment.
FAQ
Can AI decide the disposition of a nonconforming product?
AI can assemble and cross-check evidence and retrieve comparable cases. Keep the final disposition and release with authorized people who understand product risk and customer obligations. A “no issue found” model response is not proof of physical segregation, approval or reinspection.
Can an MRB support system live only in QMS?
QMS may store the case and approvals. If MES can advance the lot or ERP can allocate and ship it, those execution systems also need synchronized hold and release controls.
What matters most in FAT and SAT?
The central acceptance rule is that a missing condition prevents release. FAT tests decision logic and failure paths; SAT proves the same behavior with real scanners, people, locations, networks and existing systems.
What should an AI disposition RFP specify?
State physical identity and hold granularity, document effectivity, source-linked AI findings and “cannot determine” behavior, MRB authority, release conditions, system-of-record ownership, audit history, and negative FAT/SAT cases. Ask vendors to demonstrate missing evidence and contradictory versions, not only a clean demo case.
Conclusion
AI evidence cross-checking is valuable when it gives MRB a trustworthy, revision-aware packet and explicitly identifies what is absent or inconsistent. Its safe deployment depends on physical segregation, human authority, controlled execution and reinspection, consistent QMS/MES/ERP states, and negative tests at FAT and SAT. The release gate remains a governed business decision.
If your plant is mapping an incoming or in-process hold and MRB workflow, contact TOMAS TECH at the requirements stage. We can review the existing QMS, MES, ERP and paper steps with you and identify where AI cross-checking helps and which controls should be established first.
Sources
- ISO 9001 quality management systems standard: verify the applicable edition and customer requirements for your organization.
- NIST AI Risk Management Framework and AI RMF Core: roles, documentation, testing and monitoring.
- NIST manufacturing data traceability and trustworthiness guidance: provenance and trustworthiness of source data.
- NIST IR 8536 manufacturing supply-chain traceability meta-framework: a conceptual framework for provenance and linking traceability data across manufacturing supply-chain ecosystems, published September 2026.
- IEC 62381:2024: FAT/SAT scope and responsibilities for process-industry automation systems.
The workflows, fields and cases in this article are design examples; they do not replace product-specific law, customer contracts or applicable standards.
