0

我目前的情况是，我们有一个笨重、陈旧、庞大的数据库，为我们的一两个用户不太友好的系统提供动力（这是由第三方控制的，所以我无法更改）。

展望未来，我希望将必要的数据推送到一个新的（结构更好的）数据库中，并实施一种机制来保持两者之间的数据同步。

这个旧数据库的一个古怪特征是，它不仅使用空值，还使用空字符串并将它们视为空值。

在新数据库中，我想在没有数据的地方使用空值（因为我想不出任何不这样做的充分理由）。

我的问题是，当我将数据从旧数据库提取到新数据库（使用MERGE语句）时，我使用 aNULLIF([myCol], '')来检查实际上是否有任何数据 - 如果没有，则将其视为 null。

例如，在为学生同步数据时，我打算按照以下方式做一些事情（数据库和模式的名称与问题无关，所以我将结构匿名化一点）：

USE [NewDB];

DROP TABLE IF EXISTS #myTempTable;

SELECT *
INTO #myTempTable
FROM [OldDB].[schemaName].[Students];

MERGE [schema].[Person] p
USING #myTempTable s 
      ON TRIM(s.STUD_ISN) = p.StudentDBID
--WHEN MATCHED and not equal, update
WHEN NOT MATCHED 
        THEN INSERT (
                        [Surname] ,
                        [PreferredSurname] ,
                        [FormerSurname] ,
                        [Forename] ,
                        [MiddleNames] , 
                        [PreferredForename] ,
                        [DoB] ,
                        [Gender] ,
                        [GenderIdentity] ,
                        [Title] ,
                        [Deceased] ,
                        [DeceasedDate] ,
                        [StudentDBID])
             VALUES ( 
                        NULLIF(TRIM([s].[STUD_Surname]), '') ,
                        NULLIF(TRIM([s].[STUD_Preferred_Surname]), '')  ,
                        NULLIF(TRIM([s].STUD_Former_Surname), '')  ,
                        NULLIF(TRIM([s].[STUD_Forename_1]), '')  ,
                        NULLIF(TRIM([s].[STUD_Forename_2]), '')  ,
                        NULLIF(TRIM([s].[STUD_Known_As]), '')  ,
                        [s].[STUD_DOB] ,
                        NULLIF(TRIM([s].[STUD_Gender]), '')  ,
                        NULLIF(TRIM([s].STUD_Gender_Identity), '')  ,
                        NULLIF(TRIM([s].STUD_Title), '')  ,
                        [s].STUD_Deceased ,
                        [s].STUD_Date_of_Death ,
                        TRIM([s].STUD_ISN)
                    );

现在，显然这个当前查询只处理以下场景：

旧数据库有数据，新数据库没有

因为我对以下场景的逻辑感到有些悲伤：

新数据库有相应的记录，但需要更新。

对于基于文本的字段，有 3 种不同的场景需要更新“新”数据：

  1. 两个值都不是空/空，但它们不相等
  2. “旧”数据库中的记录为空/空，而“新”数据库中的记录不是
  3. “新”数据库中的记录为空，而“旧”数据库中的记录不是

我希望做一些相对简单的事情，比如：

WHEN MATCHED AND ( 
                    (NULLIF(TRIM([s].[STUD_Surname]), '') IS NOT NULL
                        AND p.Surname IS NULL)
                    OR (NULLIF(TRIM([s].[STUD_Surname]), '') IS NULL
                        AND p.Surname IS NOT NULL)
                    OR (NULLIF(TRIM([s].[STUD_Surname]), '') != p.Surname)
                 ) -- do the same for the other columns
THEN UPDATE SET
                Surname = s.STUD_Surname

但是，NULLIF(TRIM([s].[STUD_Surname]), '') IS NOT NULL显然是无效的？

我最好的选择是在子句中换掉NULLIFfor an ...还是我错过了一些可以提高效率的东西？ISNULLWHEN MATCHED

4

3 回答 3

0

可以简化更新检查。

由于源临时表没有 NULL。
您只需要比较修剪后的值。

MERGE Person t
USING #tmpTable s 
   ON s.STUD_ISN = t.StudentDBID
WHEN NOT MATCHED THEN 
  INSERT (
    StudentDBID
  , Forename
  , PreferredForename
  , Surname
  )
  VALUES (
    NULLIF(TRIM(s.STUD_ISN), '')
  , NULLIF(TRIM(s.STUD_Forename), '')
  , NULLIF(TRIM(s.STUD_Forename_1), '')
  , NULLIF(TRIM(s.STUD_Surname), '')
  )
WHEN MATCHED
AND (
     COALESCE(t.Forename, '') != TRIM(s.STUD_Forename)
  OR COALESCE(t.PreferredForename, '') != TRIM(s.STUD_Forename_1)
  OR COALESCE(t.Surname, '') != TRIM(s.STUD_Surname)
) 
THEN 
UPDATE SET
  UpdatedOn = SYSDATETIME()
, Forename = NULLIF(TRIM(s.STUD_Forename), '')
, PreferredForename = NULLIF(TRIM(s.STUD_Forename_1), '')
, Surname = NULLIF(TRIM(s.STUD_Surname), '')
;

SELECT *
FROM Person

学生DBID | 姓名 | 首选名字 | 姓氏 | 已更新 | 创建于                  
:------------ | :------- | :---------------- | :-------- | :---------------------------- | :----------------------------
B9701 | 鲍勃 | 鲍比 | 谦虚 | 2022-01-14 12:58:05.9862269 | 2022-01-14 12:58:05.9862269
J0402 | 简 | 京东 | 能源部 | 2022-01-14 12:58:05.9862269 | 2022-01-14 12:58:05.9862269
J0504 | 吉姆 | 吉米 | 野蛮人 | 空                        | 2022-01-14 12:58:05.9862269
J7201 | 约翰 | 空              | 能源部 | 空                        | 2022-01-14 12:58:05.9862269
M0803 | 迈克 | 空              | 锤子 | 2022-01-14 12:58:05.9862269 | 2022-01-14 12:58:05.9862269

关于db<>fiddle的演示在这里

于 2022-01-14T13:02:59.213 回答
0

最后，我的解决方案是在SELECT .... Into ....块中应用各种功能。像这样：

ALTER PROCEDURE [NGSync].[spFullStudentSync]
AS
BEGIN
    SET NOCOUNT ON;

    DROP TABLE IF EXISTS #students

    SELECT [STUD_ISN],
           NULLIF(TRIM([STUD_Student_ID]), '') AS [STUD_Student_ID] ,
           NULLIF(TRIM([STUD_Surname]), '') AS [STUD_Surname] ,
           NULLIF(TRIM([STUD_Forename_1]), '') AS [STUD_Forename_1] ,
           NULLIF(TRIM([STUD_Forename_2]), '') AS [STUD_Forename_2] ,
           NULLIF(TRIM([STUD_Known_As]), '') AS [STUD_Known_As] ,
           [STUD_DOB] ,
           NULLIF(TRIM([STUD_Gender]) , '') AS [STUD_Gender],
           NULLIF(TRIM([STUD_Title]) , '') AS [STUD_Title],
           NULLIF(TRIM([STUD_Ethnicity]) , '') AS [STUD_Ethnicity],
           NULLIF(TRIM([STUD_LDDHP]) , '') AS [STUD_LDDHP],
           NULLIF(TRIM([STUD_Home_Telephone_No] ) , '') AS [STUD_Home_Telephone_No] ,
           NULLIF(TRIM([STUD_Daytime_Telephone] ) , '') AS [STUD_Daytime_Telephone] ,
           NULLIF(TRIM([STUD_Mobile_Telephone] ) , '') AS [STUD_Mobile_Telephone] ,
           NULLIF(TRIM([STUD_EMail_Address] ) , '') AS [STUD_EMail_Address] ,
           NULLIF(TRIM([STUD_Former_Surname] ) , '') AS [STUD_Former_Surname] ,
           CAST( CASE WHEN NULLIF(TRIM([STUD_Deceased] ) , '') = 'D' THEN 1 else 0 end AS bit ) AS [STUD_Deceased] ,
           NULLIF(TRIM([STUD_Deletion_Flag] ) , '') AS [STUD_Deletion_Flag] ,
           [STUD_Delete_Merge]  ,
           NULLIF(TRIM([STUD_Photo_filename] ) , '') AS [STUD_Photo_filename] ,
           NULLIF(TRIM([STUD_Nationality] ) , '') AS [STUD_Nationality] ,
           [STUD_Date_of_Entry_in_UK] ,
           NULLIF(TRIM([STUD_Student_Type_FESR] ) , '') AS [STUD_Student_Type_FESR] ,
           [STUD_School_ISN] ,
           NULLIF(TRIM([STUD_Home_LEA] ) , '') AS [STUD_Home_LEA] ,
           NULLIF(TRIM([STUD_Employer_Code] ) , '') AS [STUD_Employer_Code] ,
           NULLIF(TRIM([STUD_Religion] ) , '') AS [STUD_Religion] ,
           NULLIF(TRIM([STUD_Location] ) , '') AS [STUD_Location] ,
           [STUD_TPS_Include] ,
           [STUD_QOE_Complete] ,
           NULLIF(TRIM([STUD_UCAS_Application_Code] ) , '') AS [STUD_UCAS_Application_Code] ,
           [STUD_MIAP_Consent_Status] ,
           [STUD_MIAP_Verification_Type] ,
           NULLIF(TRIM([STUD_MIAP_Other_Verification] ) , '') AS [STUD_MIAP_Other_Verification] ,
           NULLIF(TRIM([STUD_Bank_Sort_Code] ) , '') AS [STUD_Bank_Sort_Code] ,
           NULLIF(TRIM([STUD_Bank_AC_No] ) , '') AS [STUD_Bank_AC_No] ,
           [STUD_Bank_ISN] ,
           NULLIF(TRIM([STUD_Bank_Postcode] ) , '') AS [STUD_Bank_Postcode] ,
           NULLIF(TRIM([STUD_Bank_AC_Holders_Name] ) , '') AS [STUD_Bank_AC_Holders_Name] ,
           NULLIF(TRIM([STUD_Current_Tutor_Group] ) , '') AS [STUD_Current_Tutor_Group] ,
           NULLIF(TRIM([STUD_Current_PostCode] ) , '') AS [STUD_Current_PostCode] ,
           NULLIF(TRIM([STUD_Doctor_Name] ) , '') AS [STUD_Doctor_Name] ,
           NULLIF(TRIM([STUD_Doctor_Telephone] ) , '') AS [STUD_Doctor_Telephone] ,
           [STUD_ULN]  ,
           NULLIF(TRIM([STUD_College_Email_Address] ) , '') AS [STUD_College_Email_Address] ,
           [STUD_Date_Due_to_Leave_UK] ,
           [STUD_UK_Residence] ,
           NULLIF(TRIM([STUD_Parish] ) , '') AS [STUD_Parish] ,
           NULLIF(TRIM([STUD_Area] ) , '') AS [STUD_Area] ,
           [STUD_Bus_Pass] ,
           NULLIF(TRIM([STUD_Bus_Route] ) , '') AS [STUD_Bus_Route] ,
           [STUD_Eng_1st_Lang] ,
           NULLIF(TRIM([STUD_Language] ) , '') AS [STUD_Language] ,
           [STUD_High_Achiever] ,
           [STUD_Is_Staff] ,
           [STUD_Is_Staff_ISN] ,
           [STUD_Excluded] ,
           [STUD_RUI_3_no_contact] ,
           [STUD_RUI_1_courses] ,
           [STUD_RUI_2_surveys] ,
           [STUD_PMC_C1_post] ,
           [STUD_PMC_C2_phone] ,
           [STUD_PMC_C3_email] ,
           [STUD_Created_Date] ,
           NULLIF(TRIM([STUD_Created_User] ) , '') AS [STUD_Created_User] ,
           NULLIF(TRIM([STUD_Created_Prog] ) , '') AS [STUD_Created_Prog] ,
           [STUD_Modified_Date] ,
           NULLIF(TRIM([STUD_Modified_User] ) , '') AS [STUD_Modified_User] ,
           NULLIF(TRIM([STUD_Modified_Prog] ) , '') AS [STUD_Modified_Prog] ,
           [STUD_OK_to_use_Image] ,
           NULLIF(TRIM([STUD_Sexual_Orientation] ) , '') AS [STUD_Sexual_Orientation] ,
           NULLIF(TRIM([STUD_Gender_Identity] ) , '') AS [STUD_Gender_Identity] ,
           [STUD_Visa_Proof_Produced] ,
           [STUD_Visa_Proof_Produced_Date] ,
           NULLIF(TRIM([STUD_Visa_Proof_Produced_Details] ) , '') AS [STUD_Visa_Proof_Produced_Details] ,
           NULLIF(TRIM([STUD_Visa_Type] ) , '') AS [STUD_Visa_Type] ,
           [STUD_Visa_Expiry_Date] ,
           [STUD_Visa_Letter_Issued] ,
           [STUD_Asylum_Seeker] ,
           [STUD_Refugee] ,
           [STUD_Entered_UK_for_Education] ,
           [STUD_Restrictions_on_Stay] ,
           NULLIF(TRIM([STUD_Add_To_Portal_Title_Bar] ) , '') AS [STUD_Add_To_Portal_Title_Bar] ,
           [STUD_RUI_5] ,
           NULLIF(TRIM([STUD_Visa_Reference]) , '') AS  [STUD_Visa_Reference],
           NULLIF(TRIM([STUD_Visa_Note] ) , '') AS [STUD_Visa_Note] ,
           [STUD_FA_Bank_ISN] ,
           NULLIF(TRIM([STUD_FA_Bank_Sort_Code] ) , '') AS [STUD_FA_Bank_Sort_Code] ,
           NULLIF(TRIM([STUD_FA_Bank_AC_No] ) , '') AS [STUD_FA_Bank_AC_No] ,
           NULLIF(TRIM([STUD_FA_Bank_AC_Holders_Name]) , '') AS  [STUD_FA_Bank_AC_Holders_Name],
           NULLIF(TRIM([STUD_Marital_Status] ) , '') AS [STUD_Marital_Status] ,
           NULLIF(TRIM([STUD_Country_of_Birth] ) , '') AS [STUD_Country_of_Birth] ,
           NULLIF(TRIM([STUD_On_Supervision] ) , '') AS [STUD_On_Supervision] ,
           [STUD_S_IraqDEAS] ,
           [STUD_S_Pending_SCN] ,
           NULLIF(TRIM([STUD_Primary_LLDDCode] ) , '') AS [STUD_Primary_LLDDCode] ,
           NULLIF(TRIM([STUD_A2C_Language] ) , '') AS [STUD_A2C_Language] ,
           NULLIF(TRIM([STUD_Signature_Filename] ) , '') AS [STUD_Signature_Filename] ,
           [STUD_S_ILA_Expiry_Date] ,
           [STUD_Date_Of_Death] ,
           [STUD_S_SQA_Exception] ,
           NULLIF(TRIM([STUD_S_SQA_VID] ) , '') AS [STUD_S_SQA_VID] ,
           [STUD_S_SQA_Registration_Date] ,
           NULLIF(TRIM([STUD_FA_Bank_AC_Ref] ) , '') AS [STUD_FA_Bank_AC_Ref] ,
           NULLIF(TRIM([STUD_Bank_AC_Ref] ) , '') AS [STUD_Bank_AC_Ref] ,
           [STUD_Anonymised] ,
           NULLIF(TRIM([STUD_Quarantine_Status] ) , '') AS [STUD_Quarantine_Status] ,
           [STUD_KeycloakID] ,
           NULLIF(TRIM([STUD_Preferred_Surname] ) , '') AS [STUD_Preferred_Surname] ,
           [STUD_Exclude_From_Balancing] ,
           NULLIF(TRIM([STUD_Corresp_Preference] ) , '') AS [STUD_Corresp_Preference] ,
           NULLIF(TRIM([STUD_HESA_ID] ) , '') AS [STUD_HESA_ID] ,
           NULLIF(TRIM([STUD_FEPUS_ID] ) , '') AS [STUD_FEPUS_ID] ,
           NULLIF(TRIM([STUD_PEV_PIN] ) , '') AS [STUD_PEV_PIN] ,
           NULLIF(TRIM([STUD_Photo_GUID_Filename] ) , '') AS [STUD_Photo_GUID_Filename]
    INTO #students
    FROM [NG].[dbo].[STUDstudent];

    MERGE [people].[Person] AS [p]
    USING #students AS [s]
        ON [s].[STUD_ISN] = [p].[StudentDBID]
    WHEN MATCHED
            AND (
                    [s].[STUD_Surname] != [p].[Surname]
                        OR [s].[STUD_Preferred_Surname] != [p].[PreferredSurname]
                        OR [s].[STUD_Former_Surname] != [p].[FormerSurname]
                        OR [s].[STUD_Forename_1] != [p].[Forename]
                        OR [s].[STUD_Forename_2] != [p].[MiddleNames]
                        OR [s].[STUD_Known_As] != [p].[PreferredForename]
                        OR [s].[STUD_DoB] != [p].[DoB]
                        OR [s].[STUD_Gender] != [p].[Gender]
                        OR [s].[STUD_Gender_Identity] != [p].[GenderIdentity]
                        OR [s].[STUD_Title] != [p].[Title]
                        OR [s].[STUD_Deceased] != [p].[Deceased]
                        OR [s].[STUD_Date_of_Death] != [p].[DeceasedDate]
                        OR [s].[STUD_Is_Staff_ISN] != [p].[StaffDBID]
                )
       THEN UPDATE SET
                            [Surname] = [s].[STUD_Surname] ,
                            [PreferredSurname] = [s].[STUD_Preferred_Surname] ,
                            [FormerSurname] = [s].[STUD_Former_Surname],
                            [Forename] = [s].[STUD_Forename_1] ,
                            [MiddleNames] = [s].[STUD_Forename_2] ,
                            [PreferredForename] = [s].[STUD_Known_As],
                            [DoB] = [s].[STUD_DoB] ,
                            [Gender] = [s].[STUD_Gender] ,
                            [GenderIdentity] = [s].[STUD_Gender_Identity] ,
                            [Title] = [s].[STUD_Title] ,
                            [Deceased] = [s].[STUD_Deceased],
                            [StaffDBID] = [s].[STUD_Is_Staff_ISN]
    WHEN NOT MATCHED
        THEN INSERT (
                        [Surname] ,
                        [PreferredSurname] ,
                        [FormerSurname] ,
                        [Forename] ,
                        [MiddleNames] ,
                        [PreferredForename] ,
                        [DoB] ,
                        [Gender] ,
                        [GenderIdentity] ,
                        [Title] ,
                        [Deceased] ,
                        [DeceasedDate] ,
                        [StudentDBID] ,
                        [StaffDBID])
             VALUES (
                        [s].[STUD_Surname] ,
                        [s].[STUD_Preferred_Surname] ,
                        [s].[STUD_Former_Surname] ,
                        [s].[STUD_Forename_1] ,
                        [s].[STUD_Forename_2] ,
                        [s].[STUD_Known_As]  ,
                        [s].[STUD_DOB] ,
                        [s].[STUD_Gender] ,
                        [s].[STUD_Gender_Identity]  ,
                        [s].[STUD_Title] ,
                        [s].[STUD_Deceased] ,
                        [s].[STUD_Date_of_Death] ,
                        [s].[STUD_ISN] ,
                        [s].[STuD_IS_Staff_ISN]
                    );

END

它可能会变得更高效，但考虑到它会成为夜间任务的一部分并在人们不使用系统时运行，它在约 2 秒内处理约 35000 条记录，因此它“足够高效”。

于 2022-01-18T11:02:20.403 回答
-2

您可以使用 SQL 中可用的 Coalesce 函数来实现所需的结果

它将从传递的参数中返回第一个 Non-Null 值

SELECT COALESCE（NULL，'第一个非空'，'Alex'）

COALESCE 关键字

于 2022-01-14T10:11:31.183 回答