What if suppliers have different names for the same products?

Reasons for discrepancies in product names

Understanding the reasons for the differences helps you choose the right matching strategy.

Technical reasons

• Different ERP systems: Each supplier uses its own accounting system with its own rules for forming names
• Manual data entry: Operators enter product characteristics differently
• Import from manufacturers: different versions of catalogs and translations
• Automatic transliterations: conversion of names from other languages

Linguistic differences

• Synonyms: "mobile phone", "smartphone", "cellular"
• Abbreviations: "256GB" vs "256GB" vs "256GB"
• Word order: "Samsung Galaxy S24 Ultra 256GB Black" vs "Galaxy S24 Ultra 256GB Samsung Black"
• Register: "iPhone 15 Pro" vs "IPHONE 15 PRO" vs "iphone 15 pro"
• Separators: "iPhone15Pro" vs "iPhone 15 Pro" vs "iPhone-15-Pro"

Regional features

• Localization: European names may differ from Asian versions
• Color options: "Titanium" vs "Titanium" vs "Titanium"
• Marketing names: different suppliers add their own accents
• Units of measurement: liters vs. l, kilograms vs. kg

Methods of product comparison

There are several basic product identification methods that can be combined to achieve maximum accuracy.

1. Comparison by manufacturer's article numbers

Principle: using a unique manufacturer code (SKU, MPN - Manufacturer Part Number) as the primary identifier.

Advantages:

• High precision - the manufacturer's part number is unique
• Does not depend on the product name
• Standardized for most brands
• Easy to automate

Problems:

• Not all suppliers provide the manufacturer's part number.
• There may be typos in the article numbers.
• Different recording formats (with spaces, hyphens)
• Some products do not have article numbers (noname products)

2. Matching by barcodes

International barcode standards:

• EAN-13: European standard, 13 digits (e.g. 5901234123457)
• UPC-A: American standard, 12 digits
• ISBN: for books, 10 or 13 characters
• GTIN: global trade item identifier

Advantages of barcodes:

• Absolute uniqueness for each product item
• An international standard recognized by all market participants
• Used in logistics and warehouse accounting
• Easy to verify through open databases

Restrictions:

• Not all suppliers include barcodes in their price lists.
• One product can have several barcodes (for different regions)
• Small producers may not register barcodes
• Technical errors may occur when entering long digital codes.

3. Comparison by characteristics

When article numbers and barcodes are not available, key product characteristic analysis is used.

Key parameters for electronics:

• Brand: Apple, Samsung, Sony (required)
• Model: iPhone 15 Pro, Galaxy S24 Ultra (required)
• Memory capacity: 128GB, 256GB, 512GB
• Color: Black, White, Titanium
• Additional options: processor generation, screen size

For clothes and shoes:

• Brand
• Collection article
• Size (EU, US, UK)
• Color
• Season/Collection

4. Combined approach

The most effective strategy is to use multiple methods with prioritization:

1. First level: Search by EAN/UPC barcode (100% accuracy)
2. Second level: Search by manufacturer's part number (95-98% accuracy)
3. Third level: comparison by characteristics (85-90% accuracy)
4. Fourth level: fuzzy text comparison of names (70-80% accuracy)
5. Manual check: for the remaining cases

Automatic matching

Modern data management systems use complex algorithms to automate the matching process.

Fuzzy comparison algorithms

Levenshtein Distance:

Measures the minimum number of operations (insert, delete, replace character) to transform one string into another.

Other algorithms:

• Jaro-Winkler Distance: takes into account the position of matching characters, works better with short strings
• Soundex / Metaphone: phonetic comparison, useful for transliterations
• N-grams: comparison of fixed-length substrings
• TF-IDF: vector representation of text for searching similar documents

Tokenization and normalization

Data preparation stages:

1. Convert to lowercase: "iPhone" → "iphone"
2. Removing special characters: "iPhone-15" → "iphone 15"
3. Normalize spaces: "iPhone 15" → "iphone 15"
4. Unification of units of measurement: "256 GB" → "256gb"
5. Removing stop words: "new", "original", "official"
6. Transliteration: "iPhone" → "iphone"

Using AI and Machine Learning

Modern approaches using artificial intelligence significantly improve the accuracy of matching.

Supervised Learning

The system is trained on labeled data - pairs of products that have been manually compared.

Process:

1. Collection of a training sample (1000-5000 pairs of products)
2. Feature extraction: name similarity, article matching, category
3. Classification model training (Random Forest, Gradient Boosting)
4. Validation on a test sample
5. Applying the model to automatic matching

Deep Learning Using Embeddings

An advanced technique using neural networks to represent products in vector space.

• Word2Vec / FastText: vector representations of words and phrases
• BERT / Transformers: contextual understanding of product names
• Siamese Networks: neural networks for determining the similarity of pairs of objects
• Cosine Similarity: measuring the similarity of vectors in a multidimensional space

Practical implementation

Automatic matching process:

1. Getting a new price list: downloading a file from a supplier
2. Preprocessing: parsing, data normalization
3. Search for exact matches: by EAN, manufacturer's article number
4. Fuzzy matching: for goods without precise identifiers
5. ML analysis: application of the trained model
6. Formation of results: automatically matched + candidates for verification
7. Logging: saving all transactions for audit

Manual adjustments and exception management

Even the most advanced automation systems require manual oversight to handle non-standard cases and improve accuracy.

When manual intervention is necessary

• New brands: the system has not yet been trained on these products
• Complex configurations: products with many variations
• Similar models: iPhone 15 vs. iPhone 15 Plus: The Differences Are Critical
• Regional versions: EU version vs US version with different specifications
• Sets and kits: individual product vs. set
• Low system confidence: similarity below the established threshold

Manual matching interface

Key UI elements:

• List of candidates: items from the new price list that require verification
• System suggestions: Automatically found similar products with a similarity percentage
• Side-by-side comparison: table with characteristics of both positions
• Change history: previous comparisons of this product
• Quick Actions: accept / reject / create a new item

Managing aliases

The alias system allows you to link different names of a single product:

• Main name: "Apple iPhone 15 Pro 256GB Titanium"
• Aliases:
  • "iPhone 15 Pro 256GB Titanium"
  • iPhone 15 Pro 256GB Titanium
  • iPhone 15 Pro 256GB Titanium
  • "APPLE IPHONE 15 PRO (256GB) Natural Titanium"

The next time you import a price list with any of these names, the system will automatically recognize the product.

Dictionaries and reference books

Maintaining reference books helps standardize data:

Brand Dictionary

• "Apple" = ["Apple", "APPLE", "apple", "Apple", "Apple"]
• "Samsung" = ["Samsung", "SAMSUNG", "Samsung", "Samsung Electronics"]

Dictionary of colors

• "Black" = ["Black", "Черный", "Черный", "BLK", "Noir"]
• "White" = ["White", "White", "WHT", "Blanc"]

Units of measurement

• "GB" = ["GB", "Gb", "GB", "Gb", "gb", "gigabyte"]
• "kg" = ["kg", "kg", "kilogram", "kilogram"]

Training the system on manual solutions

Each manual comparison is valuable data for improving the ML model:

1. Logging decisions: recording of all manual comparisons
2. Periodic retraining: once a month/quarter
3. A/B testing: testing a new model on historical data
4. Feedback loop: the system becomes more accurate with each decision

Best practices and recommendations

Standardization of data at the source

Work with suppliers to improve the quality of their prices:

• Request manufacturer part numbers and EAN codes
• Agree on a standard naming format
• Ask to highlight characteristics in separate columns
• Provide suppliers with a price list template

Regular audit

Check the quality of the comparison periodically:

• Weekly: view new automatic matches
• Monthly: analysis of statistics (percentage of automation, errors)
• Quarterly: full catalog audit for duplicates
• In case of changes: verification after adding a new supplier

Performance metrics

Track key metrics:

• Percentage of automation: how many products are compared without human intervention (target: 85-90%)
• Matching accuracy: percentage of correct decisions (target: 98%+)
• Processing time: average time for manual matching of one product
• Number of takes: percentage of duplicate cards in the catalog (goal:<1%)

Conclusion

The problem of matching products with different names is an inevitable challenge when working with multiple suppliers. However, modern technologies allow for automation of up to 85-90% of the process, leaving only the most complex cases for manual processing.

Key findings

• A layered approach works best: Start with exact identifiers (EAN, article numbers), then move on to fuzzy comparison algorithms
• Machine Learning improves accuracy: Machine learning systems match 10-15% more accurately than rule-based algorithms.
• Data quality is critical: Working with suppliers on standardization pays off in the reduction of manual labor
• Manual adjustment is required: Even the best systems require human oversight for edge cases.
• The system learns from solutions: Each manual comparison makes the algorithms more accurate.

Implementation plan for the matching system

1. Analysis of the current situation: Assess the scale of the duplicate problem in the catalog
2. Data standardization: Request article numbers and EAN codes from suppliers
3. Selection of tools: Determine which algorithms are suitable for your catalog
4. Pilot project: start with 1-2 main suppliers
5. Team training: prepare employees to work with the system
6. Gradual scaling: add other suppliers
7. Continuous improvement: Analyze metrics and optimize processes

A properly configured matching system is an investment that pays for itself within months by saving staff time, reducing catalog errors, and improving the quality of data for business decision making.