Bulk Adding Products: How to Upload Thousands of SKUs in Minutes
-
Galina Ostrachinyna
Copywriter Elbuz
Launching a new online store with a catalog of 5,000 products, adding inventory from a new supplier with 3,000 SKUs, and migrating from one platform to another with 50,000 items—all these tasks require mass importing. Manually adding each product would turn the process into a months-long nightmare.
In this guide, we'll cover the bulk product import process in detail, from data preparation to error handling and performance optimization. You'll learn how to upload tens of thousands of SKUs in minutes, avoid common issues, and ensure catalog data integrity.
When is bulk import needed?
Bulk import becomes a necessity in several key scenarios:
1. Launching a new store
Initial catalog loading is the most common use case for bulk import.
- Volume: from 500 to 100,000+ products depending on the niche
- Source: Supplier price lists, export from legacy systems, databases
- Peculiarity: All data needs to be loaded at once, including images, descriptions, and specifications
- Risks: There's no way to test on a small sample - import everything at once
2. Adding a new supplier
Expanding the product range through new partners.
- Volume: 1,000-10,000 new SKUs
- Problem: duplication of goods, if the supplier offers the same thing that is already in the catalog
- Solution: import with checking for existing goods by EAN/article
3. Platform migration
Transferring an online store from one CMS to another.
- Examples: OpenCart → WooCommerce, Magento 1 → Magento 2, Shopify → PrestaShop
- Peculiarity: It is necessary to save URL, SEO data, reviews, history
- Complexity: different data structures between platforms
4. Regular updates
Periodic synchronization with suppliers or marketplaces.
- Frequency: daily, weekly or in real time
- Volume: updating 10,000-50,000 positions at a time
- Requirement: minimal site downtime, incremental updates
5. Seasonal promotions
Quickly add special product selections for the holidays.
- Examples: New Year's gifts, back-to-school, Black Friday
- Volume: 500-5,000 SKU
- Urgency: needs to be downloaded within a few hours
The boundary between manual and mass import
Manual addition is optimal for up to 50 products — if you need careful control of each card, unique descriptions, professional photography.
Bulk import starting from 100+ products — time savings are becoming critical. Adding 100 products manually would take 10-15 hours, while importing would take 10-15 minutes.
Preparing data before importing
Proper data preparation is the key to a successful import. 80% of problems arise from incorrect source files.
Data Preparation Checklist
1. File structure
- ☑ Format: CSV (UTF-8), XLSX, or XML — check which one your CMS supports
- ☑ Encoding: UTF-8 or UTF-8 BOM for correct display of Cyrillic
- ☑ CSV delimiters: semicolon (;) for data in Russian/European languages
- ☑ Headlines: the first row should contain the column names
- ☑ Consistency: all rows have the same number of columns
2. Required fields
- ☑ SKU/Article: unique product identifier (there should be no duplicates)
- ☑ Title: correct title without HTML tags
- ☑ Price: in numeric format, without currency symbols (499.99, not "€499.99")
- ☑ Category: an existing category in your store or a path to creation
- ☑ Status: active/inactive (published/draft)
3. Recommended fields
- ☑ EAN/UPC: barcode for integration with marketplaces
- ☑ Brand: manufacturer of the goods
- ☑ Quantity: remaining stock
- ☑ Weight/Dimensions: to calculate shipping
- ☑ Images: URLs or file paths (comma separated for gallery)
- ☑ Description: full description of the product
- ☑ Brief description: for product cards in the catalog
- ☑ SEO fields: meta title, meta description, CNC
4. Data validation
- ☑ No blank lines: delete rows without data
- ☑ SKU uniqueness: check for duplicate items
- ☑ Correct prices: all prices > 0, no negative values
- ☑ Valid image URLs: links start with http:// or https://
- ☑ Existing categories: all the specified categories are in the system
- ☑ Correct characters: There are no special characters that could break the import.
Typical errors in source data
Problem 1: Mixed pricing formats
Wrong: €499, 1,200.50, "2500 rubles", 99.9
Right: 499.00, 1200.50, 2500.00, 99.90
Solution: Convert all prices to dotted decimal format and remove currency symbols.
Problem 2: Special characters in names
Wrong: Samsung Galaxy S24 Smartphone | 256GB | Black
Right: Samsung Galaxy S24 256GB Black Smartphone
Solution: Remove quotes, vertical bars, and double spaces.
Problem 3: Non-existent categories
Error: The product is linked to the "Smartphones" category, but the system only has "Mobile Phones"
Solution: Create all necessary categories before importing or use the exact names of existing ones.
An example of a correct CSV structure
sku;name;price;stock;category;brand;ean;image_url;status SM-S921-256-BLK;Samsung Galaxy S24 256GB Black;849.00;15;Smartphones;Samsung;8806095184234;https://cdn.shop.com/s24.jpg;active IP-15PRO-512-BLU;Apple iPhone 15 Pro 512GB Blue Titanium;1299.00;8;Smartphones;Apple;194253775349;https://cdn.shop.com/ip15.jpg;active HP-PAV-15-I5;HP Pavilion 15 Laptop i5 16GB 512GB;649.00;23;Laptops;HP;195161197042;https://cdn.shop.com/hp15.jpg;activeBulk import tools for different CMSs
Each e-commerce platform has its own tools for bulk importing. Let's look at solutions for popular CMSs.
WooCommerce (WordPress)
Built-in importer
Path: WooCommerce → Products → Import
- Format: CSV (template available for download)
- Limits: up to 5,000 products at a time (depending on hosting)
- Peculiarities: support for variations, updating existing products
- Pros: free, easy to use
- Cons: slow for large directories, limited error handling
Product Import Export for WooCommerce (plugin)
- Formats: CSV, XML, Excel
- Possibilities: Import scheduling, import via URL/FTP, logging
- Price: $69-$199/year
- Performance: up to 50,000 items
WP All Import
- Formats: CSV, XML, Excel, JSON
- Peculiarity: visual drag-and-drop field mapping
- Price: $99-$299/year (requires WooCommerce Add-On)
- Pros: the most flexible tool, support for complex structures
Shopify
Built-in import
Path: Products → Import
- Format: CSV (Shopify Strict Template)
- Limits: Up to 20,000 items per file, 50,000 variations
- Peculiarities: automatic creation of collections and tags
- Pros: reliable, good documentation
- Cons: Rigid CSV structure, cannot import meta fields
Matrixify (app)
- Formats: Excel, CSV, Google Sheets
- Possibilities: import of metadata, redirects, blogs
- Price: $20-$200/month
- Performance: up to 500,000 items
Magento 2
System → Import
- Format: CSV
- Modes: Add/Update, Replace, Delete
- Validation: Pre-check file with error report
- Peculiarity: import attributes, categories, images
FireBear Improved Import & Export
- Formats: CSV, XML, JSON, ODS, Excel
- Sources: URL, FTP, Dropbox, Google Sheets
- Automation: cron tasks, schedule
- Price: $399-$1,499
PrestaShop
Catalog → Import
- Format: CSV
- Types: Products, Categories, Customers, Addresses
- Peculiarity: Multiple values separator (default;)
- Limits: Recommended for up to 10,000 items at a time
Store Commander
- Interface: An Excel-like editor for bulk edits
- Import: CSV, Excel with advanced mapping
- Price: €99-€399
A one-stop solution: Elbuz
Platform Elbuz Works with all popular CMS through a single interface:
- Automatic file format detection
- Import to WooCommerce, Shopify, Magento, PrestaShop, OpenCart
- Batch processing of up to 1,000,000 items
- Validation and error correction before import
- Rollback in case of problems
- Detailed statistics and logging
Validating data before importing
Validating data before running an import helps avoid catalog integrity issues and save time on error correction.
Validation levels
1. Structural validation
Checking the correctness of the file format:
- Encoding: the file is read without artifacts
- Separators: consistent number of columns in all rows
- Headlines: presence of all required fields
- Syntax: Correct quotation marks, escaping special characters
2. Data typing
Matching values to expected types:
- Prices: numerical values not less than 0
- Quantity: integers not less than 0
- URL: correct link format (http/https)
- Email: valid email addresses
- Dates: correct date format
3. Business rules
Logical correctness of data:
- SKU uniqueness: no duplicate articles
- Existence of dependencies: categories, brands, attributes exist in the system
- Pricing policy: price > cost price, price range corresponds to category
- Product restrictions: weight< максимального для доставки, габариты заполнены для крупногабаритных
4. Image validation
Checking the availability and correctness of images:
- URL availability: images are loading (HTTP 200)
- Format: JPG, PNG, WebP, GIF
- Size: at least 500x500px for the main image
- File size: optimally up to 500KB
Validation tools
Online CSV validators
- CSV Lint: Checking the structure of CSV files
- CSVkit: console utility for working with CSV
- Google Sheets: open the file and check it visually
Validation scripts
Example of a Python script for price validation:
import pandas as pd import validators def validate_product_file(file_path): errors = [] # Load file df = pd.read_csv(file_path, sep=';', encoding='utf-8') # Check for required fields required_fields = ['sku', 'name', 'price', 'category'] missing_fields = set(required_fields) - set(df.columns) if missing_fields: errors.append(f"Missing fields: {missing_fields}") # Check for SKU uniqueness duplicates = df[df.duplicated('sku', keep=False)] if not duplicates.empty: errors.append(f"Duplicate SKUs: {duplicates['sku'].tolist()}") # Check prices invalid_prices = df[df['price']<= 0] if not invalid_prices.empty: errors.append(f"Некорректные цены в строках: {invalid_prices.index.tolist()}") # Проверка URL изображений if 'image_url' in df.columns: for idx, url in df['image_url'].items(): if pd.notna(url) and not validators.url(url): errors.append(f"Некорректный URL в строке {idx}: {url}") return errors # Использование errors = validate_product_file('products.csv') if errors: print("Найдены ошибки:") for error in errors: print(f"- {error}") else: print("Файл валиден, готов к импорту")Critical errors blocking import
- File structure corruption: import is not possible
- Duplicate SKUs: will lead to the rewriting of goods
- Non-existent categories: the products will not be visible in the catalog
- Incorrect prices: products will be inactive or displayed incorrectly
Recommendation: Always fix critical errors before importing. Warnings can be fixed afterward.
Handling errors during import
Even with careful data preparation, importing may encounter errors. It's important to handle them correctly.
Types of import errors
1. Parsing errors
Reasons:
- Incorrect encoding (Cyrillic is displayed as hieroglyphs)
- Inconsistent delimiters in CSV
- Unclosed quotes in fields
Solution: Resave the file in UTF-8, check the delimiters
2. Validation errors
Examples:
- "SKU already exists" - duplicate item
- "Category not found" - non-existent category
- "Incorrect price format" - the price contains letters
Solution: Correct specific lines, re-import
3. Resource errors
Problems:
- Timeout: The import was interrupted due to a time limit.
- Memory limit: not enough memory to process
- Disk space: server space is running out (especially when loading images)
Solution: Increase PHP limits (max_execution_time, memory_limit) or split the file into parts
4. Image loading errors
Reasons:
- URL unavailable (404, 500)
- Hotlink protection on the CDN side
- Incorrect image format
- The image is too large
Solution: Import products without images, then upload images separately
Error handling strategies
Strategy 1: Fail-fast
- Behavior: import aborts on first error
- Pros: critical problems are quickly identified
- Cons: No goods are imported until all errors are corrected
- Application: initial import, migration
Strategy 2: Skip errors
- Behavior: import continues, problematic lines are skipped
- Pros: the maximum amount of goods is imported
- Cons: you may miss important items
- Application: Regular updates, import from multiple suppliers
Strategy 3: Partial import
- Behavior: The product is imported with all fields filled in, but the problematic fields remain empty.
- Example: The product is added without an image if the URL is unavailable.
- Pros: all products are included in the catalog
- Cons: incomplete cards require revision
- Application: urgent launch, gradual filling
Logging and reporting
Detailed import logs are necessary for problem analysis:
Example of an import report
============================================ IMPORT REPORT =========================================== Date: 2025-10-21 14:23:15 File: suppliers_catalog.csv Lines in file: 5,243 RESULTS: ✓ Successfully imported: 5,107 products (97.4%) ✗ Skipped due to errors: 136 products (2.6%) STATISTICS: - New products created: 4,823 - Existing ones updated: 284 - Images uploaded: 18,342 ERRORS: 1. Duplicate SKUs (23 product): - Line 145: SKU "SM-S24-256" already exists - Line 891: SKU "IP-15-PRO" already exists... 2. Non-existent categories (58 products): - Line 234: Category "Smartwatches" not found - Line 456: Category "TWS Headphones" not found... 3. Image loading errors (55 products): - Line 789: https://cdn.supplier.com/img123.jpg - 404 Not Found - Line 1023: https://cdn.supplier.com/img456.jpg - Timeout... RECOMMENDATIONS: 1. Create categories: "Smartwatches", "TWS Headphones" 2. Fix duplicate SKUs in the source file 3. Check image availability on the supplier side Full log saved: /var/log/import_20251021_142315.log ========================================Automatic error correction
Some errors can be corrected automatically:
- SKU autogeneration: if missing, create from name + ID
- Creating categories: automatically create missing categories
- Replacing images: If the URL is unavailable, use a placeholder
- Price normalization: Automatically remove currency symbols, convert commas to periods
- CNC transliteration: generate URL-friendly slug from name
Rollback for critical issues
The rollback mechanism allows you to return the directory to its pre-import state in case of critical errors.
When is a rollback needed?
- Bulk rewrite of products: The import accidentally updated all products with incorrect data.
- Price error: All prices are loaded with an error (wrong currency, wrong exchange rate)
- Deleting data: Import in "replace" mode accidentally deleted existing products
- Structure breakdown: the connections between categories, attributes, and variations are broken
Rollback implementation methods
1. Backup the database before importing
Process:
- Create a full database dump before importing
- Run import
- If you have problems, restore the database from a dump.
Pros: 100% condition recovery
Cons: takes a long time for large databases (100GB+), blocks all operations
# Create a backup before import (MySQL) mysqldump -u username -p database_name > backup_before_import_20251021.sql # Recover from problems mysql -u username -p database_name< backup_before_import_20251021.sql2. Transactional import
Process:
- Begin a database transaction (BEGIN TRANSACTION)
- Perform import
- If successful, COMMIT; if an error occurs, ROLLBACK.
Pros: instant rollback, no intermediate states
Cons: requires transaction support (InnoDB in MySQL), high load on the database
3. Product versioning
Process:
- Save the current version of the products in a separate table
- Perform import
- If there are problems, restore from the versioned table
Pros: the ability to roll back only products without affecting orders
Cons: requires additional space in the database
4. Staged import
Process:
- Import products into the staging table
- Conduct validation and checks
- If successful, move the tables to production.
Pros: safe, can be tested
Cons: more difficult to implement
Rollback best practices
Backup strategy
- Always make a backup: before the first import, migration, or mass updates
- Keep multiple versions: at least 3 recent backups
- Test the recovery: Check periodically that the backup is working
- Automate: Use cron for daily backups
- Store off-server: the backup must be on a separate storage (S3, Dropbox)
Partial rollback (for goods)
If you need to roll back only some of the goods:
# Rollback specific products according to the SKU list UPDATE products SET price = backup_products.price, name = backup_products.name, stock = backup_products.stock FROM backup_products WHERE products.sku = backup_products.sku AND products.sku IN ('SKU1', 'SKU2', 'SKU3');Critical: Staging Testing
Before importing into production:
- Create a copy of the site on a staging server
- Perform an import on staging
- Check the accuracy of the data, the display of products, and the operation of filters
- Only after success in staging, import into production
High volume productivity
Importing tens of thousands of products requires optimization for acceptable speed and reliability.
Performance bottlenecks
1. File reading speed
Problem: Reading and parsing a large file can take a significant amount of time.
Solutions:
- CSV instead of Excel: CSV parses 10-20 times faster than XLSX
- Streaming parsing: read the file line by line, rather than loading the entire file into memory
- Compression: CSV.gz takes up less space and is transferred faster over the network.
2. Recording in the database
Problem: Inserting products one by one is slow (1000 products = 1000 requests = minutes).
Solutions:
- Batch insert: insert 100-500 products per request
- Prepared statements: Reduces SQL parsing overhead
- Disabling indexes: temporarily disable indexes, create after import
- Bulk insert API: use specialized database methods
3. Uploading images
Problem: Downloading and processing images is the slowest part.
Solutions:
- Asynchronous loading: First import the products, then upload the images in the background
- Parallel loading: download 10-20 images at a time
- CDN: If the images are already on a CDN, use URLs instead of local copies
- Lazy loading: Import URL only, load image on first view
4. Indexing and cache
Problem: Recalculating indexes and invalidating cache slows down import.
Solutions:
- Deferred indexing: index after import, not after each product
- Clearing cache: clear cache once after import
- Disabling hooks: temporarily disable triggers and event listeners
Optimizing batch processing
Batch size
The optimal batch size depends on the data:
- Simple goods (data only): 500-1000 per batch
- Products with variations: 200-500 per batch
- Products with images: 50-100 per batch
- Complex products (attributes, relationships): 50-200 per batch
Rule of thumb: batch size = 1-2 seconds of processing
Batch Import Example (PHP for WooCommerce)
<!--?php // Batch-импорт товаров в WooCommerce function batch_import_products($file_path, $batch_size = 500) { $handle = fopen($file_path, 'r'); $header = fgetcsv($handle, 0, ';'); $batch = []; $imported = 0; $total_time = microtime(true); while (($row = fgetcsv($handle, 0, ';'))!== false) { $product_data = array_combine($header, $row); $batch[] = $product_data; // Импортируем батч if (count($batch) --> = $batch_size){ $batch_time = microtime(true); import_batch($batch); $batch_time = microtime(true) - $batch_time; $imported += count($batch); echo "Импортировано: $imported товаров (батч за ". round($batch_time, 2). " сек)\n"; $batch = []; gc_collect_cycles(); // Очистка памяти } } // Import the remainder if (!empty($batch)) { import_batch($batch); $imported += count($batch); } fclose($handle); $total_time = microtime(true) - $total_time; echo "Total imported: $imported products in ". round($total_time, 2). " sec\n"; echo "Speed: ". round($imported / $total_time, 2). " products/sec\n"; } function import_batch($batch) { global $wpdb; // Prepare bulk insert $values = []; $placeholders = []; foreach ($batch as $product) { $placeholders[] = "(%s, %s, %f, %d, 'publish')"; $values[] = $product['sku']; $values[] = $product['name']; $values[] = $product['price']; $values[] = $product['stock']; } $query = "INSERT INTO {$wpdb->posts} (post_name, post_title, meta_price, meta_stock, post_status) VALUES ". implode(', ', $placeholders); $wpdb->query($wpdb->prepare($query, $values)); }?>Parallel processing
Multithreaded import
Split the file into parts and import in parallel:
- Split the file: 10,000 products → 10 files of 1,000
- Run multiple processes: each one processes its own file
- Combine the results: collect statistics and logs
Acceleration: 3-5x on a multi-core server
Background task queue
Use a queuing system (Redis, RabbitMQ):
- The main process reads the file and creates tasks
- Workers process tasks from the queue.
- The number of workers can be scaled
Performance comparison
| Method | 10,000 products | 100,000 products | Complexity |
|---|---|---|---|
| Sequential (1 by 1) | 45-60 minutes | 7-10 hours | Simple |
| Batch (500 each) | 5-8 minutes | 50-80 minutes | Average |
| Batch + async images | 2-3 minutes | 20-30 minutes | Average |
| Parallel (4 streams) | 1-2 minutes | 10-15 minutes | High |
| Queue + workers | 1-2 minutes | 8-12 minutes | High |
Configuring PHP for Large Imports
Edit php.ini:
max_execution_time = 3600; 1 hour memory_limit = 512M; or more for huge files upload_max_filesize = 100M post_max_size = 100M max_input_time = 600Conclusion
Bulk product import is a critical operation for online stores of any size. Proper data preparation, optimal tools, and error handling strategies transform this potentially risky process into a routine one.
Key findings
- Подготовка данных критична: 80% успеха импорта — корректно подготовленный файл. Используйте чек-лист валидации перед загрузкой.
- Выбирайте правильный инструмент: встроенные импортеры CMS подходят для базовых задач, специализированные плагины — для сложных сценариев, универсальные платформы типа Elbuz — для автоматизации.
- Валидация обязательна: проверяйте данные до импорта, а не после. Исправить 100 ошибок в файле проще, чем 100 неправильных карточек в каталоге.
- Обрабатывайте ошибки gracefully: используйте стратегию skip errors для регулярных обновлений, fail-fast для критичных импортов.
- Всегда имейте plan B: backup перед импортом, механизм rollback при проблемах, staging-среда для тестирования.
- Оптимизируйте для масштаба: batch-обработка и параллелизм ускоряют импорт в 10-50 раз. Для 100,000+ товаров — это не опция, а необходимость.
Practical recommendations
- Start small: протестируйте импорт на 100 товарах перед полной загрузкой
- Документируйте процесс: создайте внутреннюю инструкцию по импорту для вашей команды
- Автоматизируйте рутину: если импортируете регулярно, настройте автоматическую загрузку
- Мониторьте результаты: анализируйте логи импорта, находите паттерны ошибок
- Train your team: менеджеры должны понимать базовые принципы импорта
Next steps
После освоения массового импорта рассмотрите автоматизацию смежных процессов:
- Автоматическая загрузка прайсов от поставщиков по расписанию
- Объединение данных от нескольких поставщиков в единый каталог
- Синхронизация остатков в реальном времени
- Автоматическое обновление цен с учетом курсов валют и наценок
Автоматизируйте массовый импорт товаров
Platform Elbuz полностью автоматизирует процесс импорта: от получения файлов до загрузки товаров в магазин. Валидация, обработка ошибок, rollback, оптимизация производительности — все из коробки.
- Импорт до 1,000,000 товаров
- Поддержка всех популярных форматов и CMS
- Batch-обработка с оптимальной скоростью
- Автоматическое исправление типичных ошибок
- Подробная статистика и алерты
Related materials
Save a link to this article
Galina Ostrachinyna
Copywriter ElbuzThe secrets of online store automation are revealed here, like the pages of a magic book of a successful business. Welcome to my world, where every idea is the key to online effectiveness!
Discussion of the topic – Bulk Adding Products: How to Upload Thousands of SKUs in Minutes
Bulk Adding Products: How to Upload Thousands of SKUs in Minutes
There are no reviews for this product.


Write a comment
Your email address will not be published. Required fields are checked *