1

经过几天在互联网上搜索答案并尝试自己改进这一点，我终于决定寻求帮助。

我每天从一个客户那里收到一个包含大约 110 万行数据的平面文件。我使用 SSIS (SQL Server 2012) 将此数据导入暂存数据库。这只需几秒钟。数据基本上是约会信息。

平面文件中有几个字段，但我必须使用这些字段来同步报告表：

UpdateType - 包含 INSERT、UPDATE 或 DELETE。
ChangeDate - 行更改时的日期时间戳。
UniqueKey - UniqueKey + ChangeDate 为行创建唯一键

客户的要求是我按照唯一键的 ChangeDate 顺序从报告数据库中插入、更新或删除行。我无法弄清楚如何在一组中执行此操作，因此我创建了一个 while 循环，该循环需要 20 多个小时才能运行，这太长了。

这是我收到的平面文件数据的示例：

UpdateType UniqueKey ChangeDate MoreDate
INSERT 27244595 2013-09-24 08:51:48.367 同步数据如下
DELETE 27244595 2013-09-25 10:15:08.433 同步数据如下
INSERT 27244595 2013-09-25 10:15:09.990 同步数据如下
DELETE 27244595 2013-09-25 15:02:36.287 同步数据如下
INSERT 27244595 2013-09-25 15:02:36.610 同步数据如下

正如您所看到的，同一条记录被插入然后多次删除，但情况并非总是如此。在此示例数据中，只有最后一条记录应出现在报告数据库表中，安排了 1 个约会。

这是来自同一平面文件的另一个示例：

UpdateType UniqueKey ChangeDate MoreDate
INSERT 28243572 2013-09-25 10:15:08.610 同步数据如下
INSERT 28243572 2013-09-25 10:15:09.880 同步数据如下
DELETE 28243572 2013-09-25 14:01:36.210 同步数据如下
INSERT 28243572 2013-09-25 14:02:37.287 同步数据如下

在此示例中，第一条和最后一条记录应出现在报告数据库表中。有 2 次预约。还有其他时候混合中的更新。

我不创建报告，也不知道它们长什么样。

这是我为从暂存数据库同步报告数据库而编写的代码。如果您对如何改进此过程有任何建议，我欢迎他们并感谢您的帮助。

DECLARE --DECLARE SOME VARIABLES TO USE IN THE LOOP
 @UPDATETYPE VARCHAR(6) --THIS WILL BE INSERT, DELETE OR UPDATE
,@KEY INTEGER --THIS IS THE UNIQUEKEY
,@CHANGEDATE DATETIME --THIS IS THE CHANGE DATE FROM THE FLATFILE

--START A WHILE LOOP TO GO ROW BY ROW 
WHILE (SELECT COUNT(*) FROM STAGEDB.DBO.APPIONTMENTCHANGE) > 0
    BEGIN

SELECT @UPDATETYPE = (SELECT TOP 1 [UPDATETYPE] FROM STAGEDB.DBO.APPIONTMENTCHANGE 
            ORDER BY [UNIQUEKEY], [CHANGEDATE]) --GET THE UPDATE TYPE FOR THE IF STATEMENTS

SELECT @KEY = (SELECT TOP 1 [UNIQUEKEY] FROM STAGEDB.DBO.APPIONTMENTCHANGE 
         ORDER BY [UNIQUEKEY], [CHANGEDATE]) --GET THE KEY

SELECT @CHANGEDATE = (SELECT TOP 1 [CHANGEDATE] FROM STAGEDB.DBO.APPIONTMENTCHANGE 
            ORDER BY [UNIQUEKEY], [CHANGEDATE]) --GET THE CHANGEDATE

--IF THIS ROW IS AN INSERT THEN COMPLETE THIS ON THE REPORT DATABASE
IF @UPDATETYPE = 'INSERT'
BEGIN
 INSERT INTO [REPORTDB].[DBO].[APPOINTMENT]
            ([REPORTDB].[DBO].[APPOINTMENT].[UNIQUEKEY]
            ,[REPORTDB].[DBO].[APPOINTMENT].[APPOINTMENT]
            ,[REPORTDB].[DBO].[APPOINTMENT].[CLIENTLEADBK]
            ,[REPORTDB].[DBO].[APPOINTMENT].[FIRSTNAME]
            ,[REPORTDB].[DBO].[APPOINTMENT].[LASTNAME]
            ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER]
            ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER2]
            ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER3]
            ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER4]
            ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSSTREET]
            ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSCITY]
            ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSSTATE]
            ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSZIP]
            ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSCOUNTRY])
SELECT TOP 1 [UNIQUEKEY]
            ,[APPOINTMENT]
            ,[CLIENTLEADBK]
            ,[FIRSTNAME]
            ,[LASTNAME]
            ,[PHONENUMBER]
            ,[PHONENUMBER2]
            ,[PHONENUMBER3]
            ,[PHONENUMBER4]
            ,[ADDRESSSTREET]
            ,[ADDRESSCITY]
            ,[ADDRESSSTATE]
            ,[ADDRESSZIP]
            ,[ADDRESSCOUNTRY]
        FROM [STAGEDB].[DBO].[APPIONTMENTCHANGE] 
    ORDER BY [UNIQUEKEY], [CHANGEDATE];

--ONCE THE INSERT IS COMPLETED THEN DELETE THE ALREADY WORKED RECORD FROM THE STAGING DATABASE
DELETE FROM [STAGEDB].[DBO].[APPIONTMENTCHANGE] 
WHERE [UNIQUEKEY] = @KEY AND [CHANGEDATE] = @CHANGEDATE;

END

--IF THE ROW IS A DELETE REQUEST THEN COMPLETE THIS ON THE REPORT DATABASE
IF @UPDATETYPE = 'DELETE'
BEGIN

DELETE FROM [REPORTDB].[DBO].[APPOINTMENT]
WHERE [UNIQUEKEY] = @KEY AND [CHANGEDATE] = @CHANGEDATE;

--ONCE THE DELETE IS COMPLETED THEN DELETE THE ALREADY WORKED RECORD FROM THE STAGING DATABASE
DELETE FROM [STAGEDB].[DBO].[APPIONTMENTCHANGE] 
WHERE [UNIQUEKEY] = @KEY AND [CHANGEDATE] = @CHANGEDATE;

END

--IF THE ROW IS A UPDATE REQUEST DO THAT
IF @UPDATETYPE = 'UPDATE'
BEGIN

    UPDATE [REPORTDB].[DBO].[APPOINTMENT]
       SET [REPORTDB].[DBO].[APPOINTMENT].[APPOINTMENT] = B.[APPOINTMENT]
      ,[REPORTDB].[DBO].[APPOINTMENT].[CLIENTLEADBK] = B.[CLIENTLEADBK]
      ,[REPORTDB].[DBO].[APPOINTMENT].[FIRSTNAME] = B.[FIRSTNAME]
      ,[REPORTDB].[DBO].[APPOINTMENT].[LASTNAME] = B.[LASTNAME]
      ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER] = B.[PHONENUMBER]
      ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER2] = B.[PHONENUMBER2]
      ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER3] = B.[PHONENUMBER3]
      ,[REPORTDB].[DBO].[APPOINTMENT].[PHONENUMBER4] = B.[PHONENUMBER4]
      ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSSTREET] = B.[ADDRESSSTREET]
      ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSCITY] = B.[ADDRESSCITY]
      ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSSTATE] = B.[ADDRESSSTATE]
      ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSZIP] = B.[ADDRESSZIP]
      ,[REPORTDB].[DBO].[APPOINTMENT].[ADDRESSCOUNTRY] = B.[ADDRESSCOUNTRY]
      FROM [REPORTDB].[DBO].[APPOINTMENT]
INNER JOIN [STAGEDB].[DBO].[APPIONTMENTCHANGE] B 
        ON [REPORTDB].[DBO].[APPOINTMENT].[UNIQUEKEY] = B.[UNIQUEKEY]
     WHERE [REPORTDB].[DBO].[APPOINTMENT].[UNIQUEKEY] = @KEY;

--ONCE THE UPDATE IS COMPLETED THEN DELETE THE ALREADY WORKED RECORD FROM THE STAGING DATABASE
DELETE FROM [STAGEDB].[DBO].[APPIONTMENTCHANGE] 
WHERE [UNIQUEKEY] = @KEY AND [CHANGEDATE] = @CHANGEDATE;
END
    END

我宁愿拥有所有需要插入的文件，所有需要删除的文件，以及所有应该更新的文件，而不是每一个发生的记录更改，但这是我现在必须处理的。

感谢所有认真的改进想法。请提供尽可能多的解释。

4

1 回答 1

0

使用普通的 sql 命令。首先插入

insert into realtable
(field1, field2, etc)
select field1, field2, etc
from stagingtable
where idfield in
(select idfield 
from stagingtable
except 
select idfield
from realtable)

更新

update r
set field1 = s.field1
, etc
from realtable r join stagingtable s on something
where whatever

删除

delete from reatable
where idfield in
(select idfield
from staging table
where you want the record deleted from the real table)
于 2013-10-03T16:11:18.497 回答