Remove Duplicate Products From a CSV Product Catalog

A product catalog CSV accumulates duplicates over time. Supplier feeds include items you already have. Re-exports add rows you imported last quarter. Merging two catalog versions creates overlap. The result is a catalog with the same SKU listed multiple times — sometimes with slightly different prices, descriptions, or stock counts, which makes the problem harder to spot.

The CSV Deduplicator removes duplicate product rows from any CSV catalog. Choose the column that uniquely identifies a product — usually SKU, barcode, or product name — and the tool removes every row that repeats that value. One product, one row.

Choosing the Right Deduplication Key

The best deduplication key for a product catalog is the field that is supposed to be unique per product:

• SKU — the most reliable key if your catalog has consistent SKUs. Every distinct product should have exactly one SKU.
• Barcode / UPC / EAN — reliable for physical goods with manufacturer barcodes. Good for supplier feed deduplication.
• Product Name — usable if SKUs are inconsistent, but riskier. Slight name differences ("Blue T-Shirt" vs "Blue T-shirt") may prevent correct matching even though they are the same product.
• Product ID — works well if the IDs are consistent across your data sources. If they are auto-generated per export, IDs may differ for the same product.

SKU is usually the right choice. If your catalog lacks SKUs or they are inconsistent, barcode is the next best option.

Variant Products: Size, Color, and Options

A product with multiple variants (sizes, colors) often has one row per variant in a catalog CSV. This is correct structure — not a duplicate. A red size-M t-shirt and a blue size-L t-shirt should be separate rows even if they share a base product name.

The deduplication key handles this correctly if you use the variant-level SKU (which should be unique per variant) rather than the base product name (which would be the same across all variants of the same product).

If you want to deduplicate at the product level — keeping only one row per base product regardless of variants — use the base product name or parent SKU as the key, and choose "Keep first occurrence" to retain one representative row.

Sell Custom Apparel — We Handle Printing & Free Shipping

When Duplicate Rows Have Different Values

The most problematic catalog duplicates are rows that share a SKU but have different prices, stock counts, or descriptions. This happens when:

• A product was updated in one system but the old data was re-imported alongside the updated record
• Two supplier feeds include the same product with different pricing
• A manual edit created a second row instead of updating the existing one

Before running deduplication, download the duplicates file — the CSV Deduplicator lets you download the rows that were removed as a separate file. Review conflicting values and decide which version to keep. If the newer row has better data, you may want to keep the last occurrence rather than the first.

Step-by-Step: Deduplicate Your Product Catalog

1. Open the CSV Deduplicator and upload your product catalog CSV
2. Select the SKU column (or barcode, or product ID) as the matching column
3. Choose "ALL" mode if matching on a single column, or "ANY" mode to catch duplicates on either SKU or barcode
4. Click Deduplicate
5. Review the summary: how many rows were removed, how many unique products remain
6. Download the duplicates file first and review any conflicting data
7. Download the clean catalog and use it for your import or feed

The tool normalizes values before comparing, so SKUs with inconsistent spacing or capitalization ("SKU-001" vs "sku-001" vs "SKU- 001") are still caught as matches.

After Deduplication: Re-Importing the Clean Catalog

A deduplicated product catalog is ready to import into Shopify, WooCommerce, a PIM system, or any platform that accepts CSV product feeds. One common next step: renaming columns to match your platform's import format.

If your catalog column names do not match the import template (e.g., "Product Name" instead of "Title" for Shopify, or "Price" instead of "Regular price" for WooCommerce), use the CSV Column Mapper to rename and reorder columns after deduplication. You end up with a clean, correctly structured file in a single workflow — no spreadsheet required.

Frequently Asked Questions

What if my catalog has no SKU column?

Use the product name as the deduplication key. Be aware that minor spelling differences will prevent a match — "Blue T-Shirt" and "Blue T-shirt" are treated as different values. Clean up name capitalization first if consistency is an issue.

Can I deduplicate across two separate catalog CSV files?

Combine the two files into one CSV first (paste all rows together, keeping one header row), then run deduplication on the combined file. The CSV Deduplicator processes one file at a time.