mysql - Show Duplicates on MySQL bulk insert

Question

I have a CSV file which I need to enter into my Database. My modus operandi is a Bulk insert. One of the columns has a uniqueness constraint attached to it but it is not the primary column. If there is a duplicate entry, it correctly skips the line and does not enter it into the database. (On command line it indicates Duplicates: n, where n is the total number of duplicates).

Is there anyway I can retrieve the duplicate row numbers ? For instance, using Show Warnings or Show Errors, it states the last MySQL errors and warnings, is there anyway I can retrieve the duplicates from MySQL alone ?

Thanks.

Answer 1

You could enter the data into a temporary table first, without the uniqueness constraint, and perform a query to find all the duplicates.

SELECT unique_column, count(*) c
FROM temp_tablename
GROUP BY unique_column
HAVING c > 1;

Then copy it from the temporary table to the real table with:

INSERT IGNORE INTO tablename SELECT * FROM temp_tablename;