If your factory is considering AI duplicate detection for its item master, start at the decision just before a new item number is approved. Duplicate records fragment purchasing history and inventory visibility. Yet merging two similar names when the specifications or units differ can cause a much more serious error. This guide explains how a Thai manufacturer can add candidate search, human disposition, exceptions and an audit trail while retaining its ERP as the system of record.
Prevent duplicate item registration before approval
A duplicate found after creation may already be referenced by quotations, purchase orders, stock, bills of materials and work instructions. Deleting one number is rarely a complete remedy. At application time, the requester can instead see existing candidates and choose whether to reuse an item, justify a separate item, or hold the request for clarification. A similarity score is evidence for review, not automatic authority to merge records.
SAP’s Material Master Data Governance documentation describes duplicate checking, approval workflows and a document trail. Oracle Product Hub describes matching during item creation, update and import, and showing candidates to a data steward for a deliberate decision. These are examples of specific product capabilities; verify the version and configuration in your own environment. Proposed AI-based semantic ranking should help find candidates missed by conventional search. It should not replace approval control.
Define one registration request as one review case. Collect the original item name, class, manufacturer and part number, drawing or specification, dimensions, units, optional GTIN, site and intended use. Search existing records and show candidates with differences. The requester chooses reuse, new registration, or hold. A designated reviewer confirms the reason before ERP assigns a new number. Merging or retiring existing records belongs to a separate controlled process.
Keep the registration gate narrower than an enterprise MDM program
Enterprise master data management may also cover suppliers, customers, BOMs, migration and multiple sites. A duplicate prevention pilot does not need to finish all those tasks first. It does need a defined item class, data owner, application and approval states, and one authority for ERP writes. Our manufacturing master data management guide covers the wider program. Here the operational question is whether a new item number should be issued.
Specify the data needed for an item decision
Name alone is insufficient. Mandatory fields should differ by item class. A machined component may need manufacturer, model, size, tolerance, material and drawing revision. A chemical may need concentration, package, grade and storage conditions. Purchasing and base units, box versus piece, and legitimate conversions must be visible. Replacing unit strings with a generic synonym can conceal a real difference.
A useful input contract contains request ID, requester, intended use, class, original and normalized name, manufacturer, manufacturer part number, specification attributes, base and transaction units, optional GTIN, drawing or datasheet link, site and search results. Return incomplete applications for completion instead of treating missing data as “no match.” Missing identifiers make duplicates harder to see.
| Attribute | Check before approval | Unsafe shortcut |
|---|---|---|
| Name and aliases | Language, abbreviations and word order | Equating similar names |
| Manufacturer and part number | Aliases and separator conventions | Treating the same number from two manufacturers as identical |
| Specification and drawing | Dimension, material, rating and revision | Ignoring a critical difference because text is similar |
| Units and pack | Base unit, purchasing unit and conversion | Comparing a box directly with a piece |
| GTIN | Issuer, packaging level and stored value | Ignoring missing or different packaging data |
| Site and lifecycle status | Active, blocked, obsolete or site-specific | Reusing obsolete stock codes as if active |
Oracle documents GTIN, trading partner item numbers and attribute matching, including configuration-specific behavior around units and item classes. Use this as a requirements prompt, not a universal promise about every ERP. GS1 defines GTIN as an identifier for trade items. A GTIN may be a strong candidate key when present, but many internal items do not have one, and packaging levels and data quality still matter.
Separate “same item,” “approved substitute” and “looks similar”
The same manufacturer’s model can have revisions or applicability differences. Different models can be approved substitutes. Show at least three candidate types: likely identical record, approved substitute, and related item for reference. Semantic proximity does not prove interchangeability. Engineering and quality decide whether a substitute is permissible. Master record consolidation and sourcing substitution are different decisions.
Rules for new item numbers during engineering change must also be explicit. Our PLM–ERP engineering change guide covers revision, effectivity and work in progress. The registration gate should read those rules rather than labeling a close specification as an existing item on its own.
Use three layers for candidate detection
Layer one checks defined exact keys: existing item number, GTIN, manufacturer plus part number, or drawing plus revision. Layer two normalizes punctuation, whitespace, unit notation, manufacturer aliases and class, then compares strings and structured attributes. Layer three uses semantic retrieval or learned ranking to bring in records with different wording but similar use or description. Keep exact and attribute evidence alongside AI rankings.
“AI” need not mean one large language model. Embeddings, learned ranking and maintained synonym lists are different options. Retain deterministic rules where they are more reproducible. IBM’s probabilistic matching explanation separates standardization, candidate selection and comparison. Its person or organization thresholds should not be copied into manufacturing item decisions; the design principle is the separation of these steps.
Show why each candidate appeared. A review screen might say “manufacturer part number matches,” “base unit differs,” “temperature rating missing” and “old drawing obsolete.” A high semantic score should never hide a safety-critical specification conflict. Too few candidates miss duplicates; too many cause reviewers to skim. Set list length and thresholds from actual applications and review workload.
A similarity score is not automatically a probability of identity
A score of 0.9 does not necessarily mean a 90% probability that two items are identical. Display exact identifiers, specification conflicts, unit differences and text similarity separately. If score calibration is needed, use previously adjudicated cases and evaluate by item class. A vendor’s headline “accuracy” is not an acceptance criterion without a defined test set.
False positives can lead to ordering or using the wrong item. False negatives preserve duplicate records, split stock and obscure purchase volume. The cost of each error differs by item class. Safety-critical items may need more human review; lower-risk consumables may allow a lighter route with recorded justification. Measure both error types and the review burden instead of choosing one global threshold.
Let people decide and record exceptions
Store business outcomes such as reuse existing item, approve new item, hold for specification, send to substitute review, or return to requester. Assign rights to the requester, master data steward, engineering, quality and purchasing by class and risk. A new record despite a candidate need not be forbidden, but the reason and approver must be recorded.
Useful exception reasons include customer-specific item, preventing confusion with an obsolete revision, different package unit, different specification despite the same manufacturer, and incorrect candidate data. If “other” dominates, improve the list or the search. Reuse returns the existing item number. Only approved new registration invokes ERP creation. Handle a lost ERP response without leaving an application marked completed while no item number exists.
| State | Human action | Evidence | ERP action |
|---|---|---|---|
| Application | Submit required attributes and evidence | Request ID, original values, requester, time | None |
| Candidate review | Inspect matches and differences | Search version, candidate IDs, reasons, scores | None |
| Disposition | Reuse, create or hold | Reviewer and exception reason | Return existing ID if reused |
| Approval | Authorized reviewer confirms | Approver and change history | Request creation if new |
| Confirmation | Reconcile ERP response | ERP item ID, response ID, result | Suppress duplicate retries |
The audit trail should retain original and changed inputs, the master snapshot or search index version, matching rules or model version, displayed candidates, decision, exception, approver and ERP result. Model updates can change ranking, so reviewers must be able to reconstruct what was shown at the time. Set retention and access under company policy. Check data transfer, storage and permissions before sending drawings or pricing to an external AI service.
Integrate with the ERP duplicate check
If the ERP already provides approvals, respect its states and data ownership. A search service beside an application form, an ERP extension, or a portal that submits approved requests can all work. Decide which system issues the definitive item number. If AI uses a temporary number, define mapping to the final number and failure recovery. Use the application ID as an idempotency key. After an ambiguous response, query ERP before retrying creation.
Decide whether searches include obsolete, blocked and site-specific records. Hiding obsolete items can produce a new duplicate; presenting them as reusable can cause an invalid purchase. Show status clearly and route decisions to reuse, reactivate or create separately. Account for search-index delay: if updates run overnight, a newly approved item may be absent from later searches. Include approved pending requests in candidate search and recheck exact keys before final ERP creation.
Two people may request the same part simultaneously. Candidate suggestions alone cannot prevent simultaneous commits. Test concurrency, ERP uniqueness rules and recovery. Start data cleaning with one class: manufacturer aliases, part number conventions, units and lifecycle status. Send mistakes in existing records to a separate correction ticket, with an owner and approval. Do not change historical master records as a hidden side effect of candidate review.
Write an RFP around business cases, not an “AI” label
Describe ERP, sites, item classes, application volume, languages, existing screens, approval roles and data interfaces. Provide anonymized examples that preserve meaningful part and specification differences. Ask vendors to distinguish standard capability, configuration, custom development and manual work.
Request exact, attribute and semantic candidate methods; manufacturer alias management; conditions for unit conversion; GTIN and part-number handling; visible reasons and conflicts; candidate-list controls; hold and exception states; permissions; audit; ERP retry behavior; and model version control. Test mixed Japanese, English, Thai and Vietnamese names, including untranslatable codes. Translated search terms must not silently overwrite the official item name.
| RFP area | Demonstration case | Evidence to inspect |
|---|---|---|
| Exact keys | Same maker/model; same model from a different maker | Candidate and reason |
| Attributes | Different unit, pack, size, material or revision | Difference view and hold condition |
| Semantic search | Abbreviation and multilingual descriptions | Candidate ranking and explanation |
| Human decision | Justified exception after a high score | Approval path and audit log |
| ERP interface | Lost response, retry and simultaneous requests | Duplicate prevention and recovery |
| Data governance | Drawings, pricing and item data | Transfer scope and permissions |
Estimate extraction and cleanup, class rules, ERP integration, human evaluation, ongoing dictionary and model changes, and user training separately. A low software subscription may still leave substantial review work. Compare vendors against the same case set and define which work step should get faster or which error should be avoided.
Accept the whole process in FAT and SAT
Use FAT to check the agreed logic with prepared data and a simulated interface. Use SAT to confirm real site permissions, ERP connection and daily operation. Follow each case from request through item ID, not just candidate ranking. Have domain reviewers label reference cases as identical, distinct, approved alternative or insufficient information. Keep disputed cases with their rationale rather than discarding them from the evaluation.
Measure missed identical items, incorrect identity suggestions, candidates per case, exception handling time, time to ERP confirmation and duplicate creations. Set targets from the factory’s baseline and risk by class; an unsupported common “95% accuracy” target is not useful. Even a good ranking fails if the requester cannot understand the differences or ignores the list.
FAT cases should include similar names with different ratings, different names with the same maker and model, GTIN with a different pack, missing conversion, obsolete items, missing documents, mixed-language descriptions, simultaneous applications, lost ERP responses and a model update. In SAT, real requesters and approvers should test latency, shift handover, delegated approval and manual recovery. Link fixes to the changed rule or model version and retest.
Start with one site and a class where attributes can be compared. Evaluate historical cases offline, run a period showing candidates while retaining the existing approval route, and then introduce controlled live use. Record where AI changes human disposition. Expand only after checking outcomes; do not copy thresholds to a class with different specifications or consequences. In operation, track reuse, later discovered duplicates, exception reasons, returned applications and false positives by class. Review training labels so an old human mistake does not become model truth.
Handle ambiguous items and interpret results
The hardest reviews are often neither identical nor clearly distinct. A manufacturer’s successor model may differ in dimensions, electrical rating, warranty, customer approval or effectivity date. Do not treat “successor” on a candidate screen as an approved substitute unless engineering and quality have recorded that decision. Keep manufacturer, supplier, internal and customer part numbers in separate fields so the reviewer knows who issued each number. The same drawing number with a different revision also needs a change assessment; a missing drawing is a reason to hold the request, not evidence of uniqueness.
If sites use different languages or purchase units, first decide whether the company requires one global item number or site-specific numbers with a documented cross-reference. Similar data does not itself settle that policy. A reviewer should see strong candidates and critical differences first, with drawings and prior decisions one click away. Show risk-relevant differences in words as well as color. Re-run matching when the requester changes a specification, class or manufacturer after a hold; other items may have been approved in the meantime.
A rising reuse rate is not automatically success: unsafe consolidation can raise it too. Inspect reuse alongside false matches, later-discovered duplicates, returns for missing data and specification corrections. Test new manufacturers, obsolete items and multilingual descriptions separately from familiar historical models. Classify misses by input quality, normalization, candidate retrieval, ranking, display and ambiguous business rules. Fix the responsible layer instead of retraining the model for every issue. If reviewers ignore good candidates because drawings or units are on another screen, improve the workflow. Record who is responsible for a long-held request and rerun the candidate check before resuming it.
Practical demonstration cases
Give vendors masked examples from your actual catalog. Put “AX-20” beside “AX20” from the same maker, then “AX-20A” with a different rating. The system should find the first pair and reveal why the third may be separate. Add a box of 100 bolts and an individual bolt, then two bolts with different lengths. Packing and physical specification must be displayed separately. For language testing, place Japanese, English and Thai names for the same proximity sensor beside a similarly described photoelectric sensor. Semantic retrieval can help locate the former but must expose the different sensor type. Finally interrupt ERP after approval, retry, and submit two concurrent requests for the same part. These cases test the registration gate as well as the search model.
Assign ownership after rollout
The item master steward maintains matching rules and exception reasons. Engineering and quality judge specification differences and substitute approval; purchasing confirms manufacturers and transaction units. IT operates the ERP interface, search index, permissions and failure alerts. If a requester cannot understand candidate differences, improve the input form and review screen before adding more rules. Every inquiry should be traceable through original values, compared attributes, human approval and the final ERP result.
The steward and IT should approve dictionary or model changes together and compare a fixed test set before and after each release. Check that a new abbreviation does not pull a genuinely different item into the same candidate group. Record each false match or miss with item class, cause, temporary action, permanent fix and retest result. Use reviewed examples to train staff and prepare expansion to the next item class.
Frequently asked questions
Is ERP standard duplicate checking enough?
It depends on product version, configuration and data quality. Verify whether your current ERP shows candidates at approval and records the outcome. Use the standard capability where it meets the need. Add semantic search only after testing an unmet problem such as multilingual descriptions.
Should part master AI automatically merge matching numbers?
No. Manufacturer, revision, rating, unit, site and status can differ. AI can find candidates and explain differences. Authorized people decide reuse or new creation. Consolidating two existing records requires a separate review of purchase orders, stock and BOM references.
Can items without GTINs be checked?
Yes. Use maker and part number, drawings, class, dimensions, material, units and names. Missing identifiers increase uncertainty. “No candidates” is not proof that there are no duplicates; return insufficient cases for completion or review.
What matters most in an item master AI RFP and FAT/SAT?
With real cases, the system must expose important specification conflicts, support a documented human disposition, and prevent duplicate ERP creation after retries. Define numeric targets from the item class and baseline, and test the full request-to-confirmation workflow.
Summary
AI duplicate detection is most useful as a pre-approval decision aid. Exact identifiers, attribute differences and semantic similarity help find candidates; people decide reuse, exceptions and new item creation. Keep unit, specification, manufacturer, GTIN, revision and lifecycle status visible. Test missed duplicates, false matches, review effort and ERP recovery with real item classes in the RFP and FAT/SAT.
If you are exploring duplicate checks for a factory in Thailand, you can contact TOMAS TECH while defining the pilot item class, application form and current ERP approval flow. We can help identify where candidate review fits and what a small evaluation should cover.
