Checking and removing duplicate attribute values via AI

What is this system for?

Imagine: your online store has thousands of products with specifications that were manually entered by different operators or downloaded from suppliers. Over time, a problem has accumulated—the same specifications are recorded differently:

• WiFi, Wi-Fi, Wi-Fi - three variants of the same thing
• "Black", "black", "BLACK" - different register
• USB Type-C, USB Type C, USB Type-C - different spaces
• 1000 mm, 100 cm, 1 m - different units of measurement

What this means for your store:

❌ The filters on the website are not working correctly—the buyer sees the same parameter multiple times

❌ Product search does not find the desired items

❌ It looks unprofessional and confuses buyers

❌ It is impossible to correctly compare products by characteristics

Our AI-powered system solves this problem automatically!

AI analyzes all attribute values in a category, finds duplicates (even if they're written differently), groups them, and suggests a single, correct value. You simply check the results and clear all product data with one click.

What will you get in the end?

• ✅ Clean data - all identical values are reduced to a single form
• ✅ The right filters — buyers see the correct list of characteristics
• ✅ Professional look — the store looks neat and reliable
• ✅ Saving time — AI does in minutes what would take hours of manual work
• ✅ Improving SEO - structured data is better indexed
• ✅ Increased conversion — customers find the product they need easier

How does this work

The system uses advanced artificial intelligence technologies to analyze text data. The AI understands the meaning of each attribute value and can determine what Wi-Fi And WiFi - it's the same thing, even if the spelling is different.

The process is completely controlled—the system only suggests options, and you always make the final decision.

Step-by-step guide

Step 1. Select a category

Open the duplicate check page: Attribute Directory → Select Category → Select Attribute → "Check for Duplicates via AI" Button

Advice: Start with a small category to get the hang of the system, then move on to larger ones.

Step 2. Category Information

On the page you will see a card with information:

• Category name — full path in the category tree
• Number of attributes — how many characteristics are checked

Step 3. Start the scan

Click the button Find Duplicates - this will launch the analysis through artificial intelligence.

What's happening:

1. The system loads all unique attribute values from the products in the category
2. Sends data to AI for analysis
3. AI groups similar values and determines the correct option
4. The results are saved and displayed on the page.

Processing time: from 10 seconds to 2-3 minutes depending on the number of attributes and values.

💡 Important: During the verification, you will see a progress bar and transaction log – keep an eye on the process!

Step 4: Viewing Results

Once the analysis is complete, the page will refresh and show:

Statistics (colored cards on top)

• Duplicate groups (blue) - how many problem areas were found
• Awaiting verification (orange) - require your attention
• Confirmed (green) - ready to use
• Total duplicates (red) - total number of incorrect values

List of duplicate groups

Each group shows:

• Attribute name — what characteristic do duplicates belong to?
• Main meaning (in large print) - the correct option that will be saved
• List of duplicates (in white frame) - incorrect options that will be replaced
• Reason from AI - why are these values grouped?
• Status - awaiting verification or confirmed

Step 5. Working with groups of duplicates

Group confirmation

If the AI has correctly identified the duplicates, click the button "Confirm" (green tick).

The card will turn light green - the group is ready for use.

Group rejection

If the AI made a mistake and the values are actually different, click "Reject" (orange cross).

The group will return to pending status - changes will not be applied.

Editing the main value

There is a pencil icon next to the main value ✏️ - click on it.

The editing window will open:

1. Please enter a new correct value
2. Click "Save"
3. The system will automatically update this value in all products!

You will see a message: "Updated (updated products: X)"

💡 When is this useful: The AI has chosen the correct spelling, but you want to use a different spelling. For example, the AI suggested "Wi-Fi," but you prefer "WiFi."

Removing individual values from a group

In the duplicate list, each value is displayed as a "chip" (plaque). Hover your cursor over it, and a cross will appear on the right. ✕.

Click on the cross:

1. A confirmation window will appear.
2. Click "Delete"
3. The value is excluded from the duplicate list.

This means: This value will not be replaced when applying changes - it will remain as is.

💡 Example: AI grouped "USB 2.0" and "USB 3.0" as duplicates, but they are actually different characteristics. Remove "USB 3.0" from the group.

Merging groups

Sometimes the AI creates several related groups that make sense to combine into one.

Example of a situation:

• Group 1: "4G (LTE: B1/B3)" → duplicates: "LTE (B1/B3/B7/B20)"
• Group 2: "4G (LTE), 3G" → duplicates: "4G and 3G (with/without 2G)"
• Group 3: "4G" → duplicates: "4G (LTE)", "3G", "2G"

You want all of these values to be in one group with the main value "4G".

How to combine:

1. In group 1, click the button "Unite"
2. A window will open with a list of other groups for this attribute.
3. Select group 3 (target)
4. Confirm the merger
5. All values from group 1 (including its main value!) move to group 3
6. Group 1 is removed

Repeat for group 2 → group 3. Eventually, all values will be in one group!

✨ This is a powerful tool. to create complex groups of duplicates when AI has separated related values.

Deleting a group

If the group is completely incorrect, click the delete button 🗑️ (basket).

The group will be deleted - no changes will be applied to the products.

Step 6. Applying changes

Once you have checked all the groups and confirmed the ones you need, click the button Apply Changes in the top panel.

What will happen:

1. The system will process all confirmed groups.
2. All duplicates in the database will be replaced with the primary values.
3. You will see a report: how many products have been updated, how many duplicates have been removed

Example message: Changes successfully applied: 47 products

⚠️ Important: Once applied, changes are irreversible! Be sure to check all groups before applying.

Step 7. Checking the result

Open several products from the category and check that the attribute values have updated correctly.

Go to the store's website and check your filters—they should now show consistent values without duplicates!

Interface Description

Top control panel

Group statuses

Color indicators in the group card

• Green check mark — the group is confirmed
• Orange watch — awaiting verification
• Green text primary value - this value is already used in product cards

Recommendations and advice

Getting ready for work

1. Make a backup copy of the database before using the system for the first time
2. Start with a test category with a small number of products
3. Check the result before mass use

Optimal sequence

1. Process the categories with the largest number of products first - get the maximum effect
2. Group similar categories together—for example, all electronics categories
3. After clearing the attributes, set up an attribute template for the category (another Elbuz tool)

Working with AI results

• Trust but verify — AI is very accurate, but can sometimes make mistakes
• Pay attention to the "AI Reason" — it explains why the values are grouped
• Use the search (if there are many groups) to quickly find the desired attribute
• Filter by status - First, look at all pending ones, then confirmed ones

When to use group merging

Combine groups when:

• AI created multiple groups for one semantic meaning (see the example with 4G above)
• You want to have one large group instead of several small ones.
• Need to add more spelling variations to an existing group

When to remove values from a group

Remove individual values when:

• AI mistakenly included a value in a group that has a different meaning
• One of the values is a valid standalone value (for example, "USB 3.0" in a group with "USB")
• The old value must be kept for compatibility with external systems.

Frequency of verification

• After mass import goods from suppliers
• Monthly for active categories (if operators fill in manually)
• Before launching advertising campaigns - Make sure the filters are working perfectly
• If problems are detected on the website (customers complain about filters)

Frequently Asked Questions

Is it possible to undo changes after applying them?

No, the changes are irreversible. Therefore, it is important to carefully check all groups before applying them. If you need protection, make a backup copy of the database.

What happens if I accidentally apply the wrong group?

Incorrect values will be replaced by the group's primary value. If this happens, you'll need to manually correct the values in the products or restore them from a backup.

The AI found too few groups. Why?

Possible reasons:

• There are really few duplicates in the category - the data is already clean
• The values are too different—the AI doesn't see the connection
• There are few products in the category with filled attributes

The AI found too many groups. What should I do?

This is normal for categories with a large number of products. Use filters and search. Process groups in stages, first the most important attributes (those used for filtering).

Is it possible to check multiple categories at once?

No, the system works with one category at a time. This is for quality control—you focus on each category separately.

Does cleaning duplicates affect SEO?

Yes, positively! Structured data is better indexed by search engines. Clean filters improve the user experience, which indirectly impacts website rankings.

Technical details

How AI identifies duplicates

The system uses a large-scale language model (LLM) trained on a massive amount of text data. The AI understands context and semantics, and can determine that values mean the same thing even when written differently.

AI grouping rules:

• Different letter cases
• Different number of spaces
• Synonyms and words with similar meanings
• Different units of measurement (if they can be converted to one)
• Different designations (WiFi / Wi-Fi)

What AI DOES NOT group:

• Different numerical values (5 and 10)
• Different colors (black and gray)
• Different versions/models (USB 2.0 and USB 3.0)

Conclusion

The AI-powered attribute duplicate checking system is a powerful tool for maintaining data quality in an online store. Regular use of the system ensures:

• ✨ Professional catalog appearance
• ✨ Correct operation of filters and search
• ✨ Convenience for buyers
• ✨ Saving operators' time
• ✨ Improving sales conversion

Clean data = happy customers = more sales!

💡 Get started right now – select a category and run your first scan. You'll be surprised how many duplicates the AI will find, and how quickly your catalog will become cleaner and more professional!