0

我有这个相当疯狂的查询，用于查找除 FIRST记录之外的所有重复值。运行 38000 条记录需要很长时间；大约 50 秒。

UPDATE exr_exrresv
    SET mh_duplicate = 1
WHERE exr_exrresv._id IN
(
     SELECT F._id
     FROM exr_exrresv AS F
     WHERE Exists 
     (
          SELECT PHONE_NUMBER, 
                 Count(_id)
          FROM exr_exrresv
          WHERE exr_exrresv.PHONE_NUMBER = F.PHONE_NUMBER 
                AND exr_exrresv.PHONE_NUMBER != '' 
                AND mh_active = 1 AND mh_duplicate = 0
          GROUP BY exr_exrresv.PHONE_NUMBER
          HAVING Count(exr_exrresv._id) > 1)
     )
AND exr_exrresv._id NOT IN
(
   SELECT Min(_id)
   FROM exr_exrresv AS F
   WHERE Exists 
   (
       SELECT PHONE_NUMBER,
       Count(_id)
       FROM exr_exrresv
       WHERE exr_exrresv.PHONE_NUMBER = F.PHONE_NUMBER 
             AND exr_exrresv.PHONE_NUMBER != '' 
             AND mh_active = 1 
             AND mh_duplicate = 0
       GROUP BY exr_exrresv.PHONE_NUMBER
       HAVING Count(exr_exrresv._id) > 1
   )
       GROUP BY PHONE_NUMBER
);

关于如何优化它或我应该如何开始它的任何提示？我已经检查了查询计划，但我真的不确定如何开始改进它。临时表？更好的查询？

这是解释查询计划的输出：

0|0|0|SEARCH TABLE exr_exrresv USING INTEGER PRIMARY KEY (rowid=?) (~12 rows)
0|0|0|EXECUTE LIST SUBQUERY 0
0|0|0|SCAN TABLE exr_exrresv AS F (~500000 rows)
0|0|0|EXECUTE CORRELATED SCALAR SUBQUERY 1
1|0|0|SEARCH TABLE exr_exrresv USING AUTOMATIC COVERING INDEX (PHONE_NUMBER=? AND mh_active=? AND mh_duplicate=?) (~7 rows)
1|0|0|USE TEMP B-TREE FOR GROUP BY
0|0|0|EXECUTE LIST SUBQUERY 2
2|0|0|SCAN TABLE exr_exrresv AS F (~500000 rows)
2|0|0|EXECUTE CORRELATED SCALAR SUBQUERY 3
3|0|0|SEARCH TABLE exr_exrresv USING AUTOMATIC COVERING INDEX (PHONE_NUMBER=? AND mh_active=? AND mh_duplicate=?) (~7 rows)
3|0|0|USE TEMP B-TREE FOR GROUP BY
2|0|0|USE TEMP B-TREE FOR GROUP BY

任何提示将不胜感激。:)

此外，我正在使用 Ruby 进行 sql 查询，所以如果逻辑离开 SQL 并用 Ruby 编写更有意义，那是可能的。

架构如下，你可以在这里使用sqlfiddle：http ://sqlfiddle.com/#!2/2c07e

_id INTEGER PRIMARY KEY
OPPORTUNITY_ID varchar(50)
CREATEDDATE varchar(50)
FIRSTNAME varchar(50)
LASTNAME varchar(50)
MAILINGSTREET varchar(50)
MAILINGCITY varchar(50)
MAILINGSTATE varchar(50)
MAILINGZIPPOSTALCODE varchar(50)
EMAIL varchar(50)
CONTACT_PHONE varchar(50)
PHONE_NUMBER varchar(50)
CallFromWeb varchar(50)
OPPORTUNITY_ORIGIN varchar(50)
PROJECTED_LTV varchar(50)
MOVE_IN_DATE varchar(50)
mh_processed_date varchar(50)
mh_control INTEGER
mh_active INTEGER
mh_duplicate INTEGER
4

2 回答 2

1

从您的帖子中猜测，您似乎正在尝试更新mh_duplicate具有相同电话号码的任何记录的列，如果它不是具有该电话号码的第一条记录？

如果这是正确的，我认为这应该让您更新 id（您可能需要添加适当的 where 条件）——从那里，更新是直截了当的：

SELECT e._Id
FROM exr_exrresv e
JOIN
 ( SELECT t.Phone_Number
    FROM exr_exrresv t
    GROUP BY t.Phone_Number
    HAVING COUNT (t.Phone_Number) > 1
  ) e2 ON e.Phone_Number = e2.Phone_Number
LEFT JOIN 
 ( SELECT MIN(t2._Id) as KeepId
    FROM exr_exrresv t2
    GROUP BY t2.Phone_Number
  ) e3 ON e._Id = e3.KeepId
WHERE e3.KeepId is null

还有SQL 小提琴。

祝你好运。

于 2013-01-21T07:30:17.793 回答
1

如果存在具有匹配 phone_number 和较小 _id 的活动记录，则认为记录重复。（无需分组或计数。）

update exr_exrresv
    set mh_duplicate = 1
where exr_exrresv._id in (
  select target._id
  from exr_exrresv as target
  where target.phone_number != ''
  and target.mh_active = 1
  and exists (
    select null from exr_exrresv as probe
    where probe.phone_number = target.phone_number 
    and probe.mh_active = 1
    and probe._id < target._id
  )
)

如果 phone_number 上存在索引，则此查询将有很大帮助，理想情况下exr_exrresv (phone_number, _id)

SQLFiddle

于 2013-01-21T08:06:55.460 回答