mysql - 大型 SQL 数据库删除重复行

Question

具有 60 000 000 条记录的大型 SQL 2008 表，我有重复行的问题。

这个命令给了我从 http://support.microsoft.com/kb/139444获取的副本

SELECT     id, sa_trvalue,  COUNT(*) AS tot  
FROM         msanal   
GROUP BY id, sa_trvalue  
HAVING      (COUNT(*) > 1)  

但是当我完成这些步骤（INTO 和 DISTINCT）时，我没有足够的内存来完成操作。

Answer 1

您可以尝试这种可能需要更少内存的方法：

WITH CTE AS
(
    SELECT  id, sa_trvalue, 
            rn = ROW_NUMBER() OVER (PARTITION BY id, sa_trvalue ORDER BY id ASC)
    FROM    msanal   
)
DELETE FROM CTE WHERE rn > 1

公用表表达式还有一个优点，即您可以轻松修改它以查看要删除的内容。因此，您只需更改DELETE为SELECT *.

Answer 2

delete msanal from msanal m1
where exists
(select null from msanal m2
where m2.sa_trvalue = m1.sa_trvalue and m2.id <> m1.id)