1

我的公司一直在数据库中收到重复记录，这些记录是在第一次创建后最多 4 分钟创建的。从逻辑上讲，一组记录由原始记录加上在该 4 分钟时间范围内创建的任何后续记录组成。初始记录的TO_DELETE值为'N'，而每个重复记录的TO_DELETE值为'Y'。每个新组都以一个'N'值重新开始。

在Deleting Invalid Duplicate Rows in SQL的帮助下，我整理了一个查询来选择它们，但它已经运行了 2 个多小时，还没有返回结果集，所以我不确定它是否陷入了无限循环. 任何帮助识别它的问题将不胜感激！

with LEAD_CTE as
(
    select *, ROW_NUMBER() over (partition by LASTNAME, FIRSTNAME, EMAIL, PRIMARY_PHONE, PROGRAMX, TERM_CODE, INQ_TYPE, LEADSOURCE order by CREATEDDATE) as ROWNUMBER
      from LEAD
     where DELETE_FLAG <> 'Y'
       and CREATEDDATE >= (GETDATE() - 7)
),
CTE as
(
    select ROWNUMBER, 'N' as TO_DELETE, CREATEDDATE, 0 as TOTAL_MINUTES, LASTNAME, FIRSTNAME, EMAIL, PRIMARY_PHONE, PROGRAMX, TERM_CODE, INQ_TYPE, LEADSOURCE
      from LEAD_CTE
     where ROWNUMBER = 1

     union all

    select l.ROWNUMBER,
           case when ((c.TOTAL_MINUTES + DATEDIFF(MINUTE, c.CREATEDDATE, l.CREATEDDATE)) > 4) then 'N' else 'Y' end as TO_DELETE,
           l.CREATEDDATE,
           case when ((c.TOTAL_MINUTES + DATEDIFF(MINUTE, c.CREATEDDATE, l.CREATEDDATE)) > 4) then 0 else (c.TOTAL_MINUTES + DATEDIFF(MINUTE, c.CREATEDDATE, l.CREATEDDATE)) end as TOTAL_MINUTES,
           l.EMAIL, l.FIRSTNAME, l.LASTNAME, l.PRIMARY_PHONE, l.PROGRAMX, l.TERM_CODE, l.INQ_TYPE, l.LEADSOURCE
      from LEAD_CTE l inner join CTE c on l.ROWNUMBER = (c.ROWNUMBER + 1)
)

  select ROWNUMBER, TO_DELETE, CREATEDDATE, TOTAL_MINUTES, LASTNAME, FIRSTNAME, EMAIL, PRIMARY_PHONE, PROGRAMX, TERM_CODE, INQ_TYPE, LEADSOURCE
    from CTE
order by LASTNAME, FIRSTNAME, EMAIL, PRIMARY_PHONE, PROGRAMX, TERM_CODE, INQ_TYPE, LEADSOURCE, CREATEDDATE

样本数据：

CREATEDDATE             | LASTNAME  | FIRSTNAME | EMAIL         | PRIMARY_PHONE  | PROGRAMX               | TERM_CODE | INQ_TYPE | LEADSOURCE
---------------------------------------------------------------------------------------------------------------------------------------------
2013-09-24 00:06:01.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:47.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:50.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:52.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:52.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:54.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:55.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:56.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2013-09-24 00:18:56.000 | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform

具有自加入功能的新 CTE：

with LEAD_CTE as
(
    select *, ROW_NUMBER() over (partition by LASTNAME, FIRSTNAME, EMAIL, PRIMARY_PHONE, PROGRAMX, TERM_CODE, INQ_TYPE, LEADSOURCE order by CREATEDDATE) as ROWNUMBER
      from LEAD
     where DELETE_FLAG <> 'Y'
       and CREATEDDATE >= (GETDATE() - 7)
)

  select l1.ROWNUMBER, l1.CREATEDDATE, l2.CREATEDDATE, DATEDIFF(MINUTE, l1.CREATEDDATE, l2.CREATEDDATE), l1.LASTNAME, l1.FIRSTNAME, l1.EMAIL, l1.PRIMARY_PHONE, l1.PROGRAMX, l1.TERM_CODE, l1.INQ_TYPE, l1.LEADSOURCE
    from LEAD_CTE l1 left join LEAD_CTE l2
      on l1.ROWNUMBER = (l2.ROWNUMBER + 1)
     and l1.LASTNAME = l2.LASTNAME
     and l1.FIRSTNAME = l2.FIRSTNAME
     and l1.EMAIL = l2.EMAIL
     and l1.PRIMARY_PHONE = l2.PRIMARY_PHONE
     and l1.PROGRAMX = l2.PROGRAMX
     and l1.TERM_CODE = l2.TERM_CODE
     and l1.INQ_TYPE = l2.INQ_TYPE
     and l1.LEADSOURCE = l2.LEADSOURCE
order by l1.ROWNUMBER

实际输出：

ROWNUMBER | CREATEDDATE             | CREATEDDATE | (no column name) | LASTNAME  | FIRSTNAME | EMAIL         | PRIMARY_PHONE  | PROGRAMX               | TERM_CODE | INQ_TYPE | LEADSOURCE
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1         | 2013-09-24 00:06:01.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
2         | 2013-09-24 00:18:47.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
3         | 2013-09-24 00:18:50.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
4         | 2013-09-24 00:18:52.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
5         | 2013-09-24 00:18:52.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
6         | 2013-09-24 00:18:54.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
7         | 2013-09-24 00:18:55.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
8         | 2013-09-24 00:18:56.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform
9         | 2013-09-24 00:18:56.000 | NULL        | NULL             | Testerson | Testy     | test@test.com | (123) 867-5309 | MS in Higher Education | NULL      | inquiry  | Webform

有趣的是，每条记录中的所有 l2 字段都以 形式出现NULL，我发现这是DATEDIFF()计算返回的结果NULL。我的预期输出是所有 l2 字段都将具有下一个 l1 记录的值，但最后一条记录的 l2 字段除外，即NULL.

4

1 回答 1

1

我认为你很接近你只需要添加

   CASE WHEN Datediff(minute, l2.createddate, l1.createddate ) > 4
                  OR l2.createddate is null
                  THEN 'Y' ELSE 'N' END,

正如我在评论中提到的那样，您需要处理加入可空字段是一件痛苦的事情

WITH lead_cte 
     AS (SELECT *, 
                Row_number() 
                  OVER ( 
                    partition BY lastname, firstname, email, primary_phone, 
                  programx, 
                  term_code, 
                  inq_type, leadsource 
                    ORDER BY createddate) AS ROWNUMBER 
         FROM   lead 
         WHERE  delete_flag <> 'Y' 
                AND createddate >= ( Getdate() - 7 )) 
SELECT l1.rownumber, 
       l1.createddate, 
       l2.createddate, 
       Datediff(minute, l2.createddate, l1.createddate ) ,
       CASE WHEN Datediff(minute, l2.createddate, l1.createddate ) > 4
                      OR l2.createddate is null
                      THEN 'Y' ELSE 'N' END,

       l1.lastname, 
       l1.firstname, 
       l1.email, 
       l1.primary_phone, 
       l1.programx, 
       l1.term_code, 
       l1.inq_type, 
       l1.leadsource 
FROM   lead_cte l1 
       LEFT JOIN lead_cte l2 
              ON l1.rownumber = l2.rownumber +1
                 AND l1.lastname = l2.lastname 
                 AND l1.firstname = l2.firstname 
                 AND l1.email = l2.email 
                 AND l1.primary_phone = l2.primary_phone 
                 AND l1.programx = l2.programx 
                 AND (l1.term_code = l2.term_code 
                       or ( l1.term_code is null and l2.term_code is null))
                 AND l1.inq_type = l2.inq_type 
                 AND l1.leadsource = l2.leadsource 
ORDER  BY l1.rownumber

演示

于 2013-09-25T21:42:49.197 回答