sql - 为什么我的分解后的 SQL 运行速度要快几个数量级？

Question

也就是说，为什么会这样：

select *
    from tableA
        /* Bunch of inner joins */
    where 
        /* Bunch of clauses */
    and (
        exists (
            select * 
                from tableB, tableC, tableD
                where (tableB.fieldNameA = 'foo') and
                   /* More clauses */
        ) or 
        exists (
            select * 
                from tableB, tableC, tableD
                where (tableB.fieldNameA = 'bar') and
                    /* More clauses */
        )
    )

运行速度比这快近 500 倍？

select *
    from tableA
        /* Bunch of inner joins */
    where
        /* Bunch of clauses */
    and exists (
        select * 
            from tableB, tableC, tableD
            where (tableB.fieldNameA = 'foo' or tableB.fieldNameA = 'bar') and
                /* More clauses */
    )

我喜欢不重复代码的想法，所以想整合到第二个版本。它们都产生相同的结果集，但第一个运行得快得多，我不能不使用它。想法？

我一直在检查查询计划，但并没有真正了解它。我确实注意到的一件事是，对于分解后的示例，聚集索引搜索[tree].[PK__tree__09746778]的实际行数接近 93,000。另一个例子没有接近这么高的行数。这是文本输出——不太确定这有多大用处。

第一个示例（已分解）产生：

|--Nested Loops(Left Semi Join, OUTER REFERENCES:([initialCategory].[inode]))
   |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[tree].[child]))
   |    |--Hash Match(Inner Join, HASH:([OPUSDev2].[dbo].[course].[inode])=([OPUSDev2].[dbo].[tree].[parent]), RESIDUAL:([OPUSDev2].[dbo].[course].[inode]=[OPUSDev2].[dbo].[tree].[parent]))
   |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[section].[inode], [Expr1037]) WITH UNORDERED PREFETCH)
   |    |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[course_term].[inode], [Expr1036]) WITH UNORDERED PREFETCH)
   |    |    |    |    |--Hash Match(Inner Join, HASH:([OPUSDev2].[dbo].[course].[course_number])=([OPUSDev2].[dbo].[course_term].[course_number]), RESIDUAL:([OPUSDev2].[dbo].[course_term].[course_number]=[OPUSDev2].[dbo].[course].[course_number]))
   |    |    |    |    |    |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[course].[PK__course__31B762FC]), WHERE:(CONVERT_IMPLICIT(tinyint,[OPUSDev2].[dbo].[course].[show_on_web],0)=(1) AND CONVERT_IMPLICIT(tinyint,[OPUSDev2].[dbo].[course].[show_on_nav],0)=(1)))
   |    |    |    |    |    |--Hash Match(Inner Join, HASH:([OPUSDev2].[dbo].[term].[term_id])=([OPUSDev2].[dbo].[course_term].[term_id]))
   |    |    |    |    |         |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[term].[PK__term__0697FACD]), WHERE:(CONVERT_IMPLICIT(tinyint,[OPUSDev2].[dbo].[term].[active_on_web],0)=(1)))
   |    |    |    |    |         |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[course_term].[course_term_pk]))
   |    |    |    |    |--Index Seek(OBJECT:([OPUSDev2].[dbo].[section].[section_idx2]), SEEK:([OPUSDev2].[dbo].[section].[course_term_inode]=[OPUSDev2].[dbo].[course_term].[inode]) ORDERED FORWARD)
   |    |    |    |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[section].[PK__section__7B264821]), SEEK:([OPUSDev2].[dbo].[section].[inode]=[OPUSDev2].[dbo].[section].[inode]),  WHERE:([OPUSDev2].[dbo].[section].[status]='Active') LOOKUP ORDERED FORWARD)
   |    |    |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[tree].[PK__tree__09746778]))
   |    |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[PK__category__2739D489] AS [initialCategory]), SEEK:([initialCategory].[inode]=[OPUSDev2].[dbo].[tree].[child]),  WHERE:([OPUSDev2].[dbo].[category].[active] as [initialCategory].[active]=(1)) ORDERED FORWARD)
   |--Concatenation
        |--Nested Loops(Inner Join)
        |    |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[PK__category__2739D489] AS [childCategory]), SEEK:([childCategory].[inode]=[OPUSDev2].[dbo].[category].[inode] as [initialCategory].[inode]) ORDERED FORWARD)
        |    |--Nested Loops(Inner Join, OUTER REFERENCES:([parentCategory].[inode]))
        |         |--Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[idx_category_2] AS [parentCategory]), SEEK:([parentCategory].[category_key]='noncredit_subjects') ORDERED FORWARD)
        |         |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[tree].[PK__tree__09746778]), SEEK:([OPUSDev2].[dbo].[tree].[child]=[OPUSDev2].[dbo].[category].[inode] as [initialCategory].[inode] AND [OPUSDev2].[dbo].[tree].[parent]=[OPUSDev2].[dbo].[category].[inode] as [parentCategory].[inode]) ORDERED FORWARD)
        |--Nested Loops(Inner Join)
             |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[PK__category__2739D489] AS [childCategory]), SEEK:([childCategory].[inode]=[OPUSDev2].[dbo].[category].[inode] as [initialCategory].[inode]) ORDERED FORWARD)
             |--Nested Loops(Inner Join, OUTER REFERENCES:([parentCategory].[inode]))
                  |--Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[idx_category_2] AS [parentCategory]), SEEK:([parentCategory].[category_key]='credit_subjects') ORDERED FORWARD)
                  |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[tree].[PK__tree__09746778]), SEEK:([OPUSDev2].[dbo].[tree].[child]=[OPUSDev2].[dbo].[category].[inode] as [initialCategory].[inode] AND [OPUSDev2].[dbo].[tree].[parent]=[OPUSDev2].[dbo].[category].[inode] as [parentCategory].[inode]) ORDERED FORWARD)

第二个例子（分解）产生：

|--Nested Loops(Left Semi Join, OUTER REFERENCES:([initialCategory].[inode]))
   |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[tree].[child]))
   |    |--Hash Match(Inner Join, HASH:([OPUSDev2].[dbo].[course].[inode])=([OPUSDev2].[dbo].[tree].[parent]), RESIDUAL:([OPUSDev2].[dbo].[course].[inode]=[OPUSDev2].[dbo].[tree].[parent]))
   |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[section].[inode], [Expr1029]) WITH UNORDERED PREFETCH)
   |    |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[course_term].[inode], [Expr1028]) WITH UNORDERED PREFETCH)
   |    |    |    |    |--Hash Match(Inner Join, HASH:([OPUSDev2].[dbo].[course].[course_number])=([OPUSDev2].[dbo].[course_term].[course_number]), RESIDUAL:([OPUSDev2].[dbo].[course_term].[course_number]=[OPUSDev2].[dbo].[course].[course_number]))
   |    |    |    |    |    |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[course].[PK__course__31B762FC]), WHERE:(CONVERT_IMPLICIT(tinyint,[OPUSDev2].[dbo].[course].[show_on_web],0)=(1) AND CONVERT_IMPLICIT(tinyint,[OPUSDev2].[dbo].[course].[show_on_nav],0)=(1)))
   |    |    |    |    |    |--Hash Match(Inner Join, HASH:([OPUSDev2].[dbo].[term].[term_id])=([OPUSDev2].[dbo].[course_term].[term_id]))
   |    |    |    |    |         |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[term].[PK__term__0697FACD]), WHERE:(CONVERT_IMPLICIT(tinyint,[OPUSDev2].[dbo].[term].[active_on_web],0)=(1)))
   |    |    |    |    |         |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[course_term].[course_term_pk]))
   |    |    |    |    |--Index Seek(OBJECT:([OPUSDev2].[dbo].[section].[section_idx2]), SEEK:([OPUSDev2].[dbo].[section].[course_term_inode]=[OPUSDev2].[dbo].[course_term].[inode]) ORDERED FORWARD)
   |    |    |    |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[section].[PK__section__7B264821]), SEEK:([OPUSDev2].[dbo].[section].[inode]=[OPUSDev2].[dbo].[section].[inode]),  WHERE:([OPUSDev2].[dbo].[section].[status]='Active') LOOKUP ORDERED FORWARD)
   |    |    |--Clustered Index Scan(OBJECT:([OPUSDev2].[dbo].[tree].[PK__tree__09746778]))
   |    |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[PK__category__2739D489] AS [initialCategory]), SEEK:([initialCategory].[inode]=[OPUSDev2].[dbo].[tree].[child]),  WHERE:([OPUSDev2].[dbo].[category].[active] as [initialCategory].[active]=(1)) ORDERED FORWARD)
   |--Nested Loops(Inner Join)
        |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[PK__category__2739D489] AS [childCategory]), SEEK:([childCategory].[inode]=[OPUSDev2].[dbo].[category].[inode] as [initialCategory].[inode]) ORDERED FORWARD)
        |--Nested Loops(Inner Join, OUTER REFERENCES:([OPUSDev2].[dbo].[tree].[parent]))
             |--Clustered Index Seek(OBJECT:([OPUSDev2].[dbo].[tree].[PK__tree__09746778]), SEEK:([OPUSDev2].[dbo].[tree].[child]=[OPUSDev2].[dbo].[category].[inode] as [initialCategory].[inode]) ORDERED FORWARD)
             |--Index Seek(OBJECT:([OPUSDev2].[dbo].[category].[idx_category_2] AS [parentCategory]), SEEK:([parentCategory].[category_key]='credit_subjects' AND [parentCategory].[inode]=[OPUSDev2].[dbo].[tree].[parent] OR [parentCategory].[category_key]='noncredit_subjects' AND [parentCategory].[inode]=[OPUSDev2].[dbo].[tree].[parent]) ORDERED FORWARD)

score 5 · Accepted Answer

确定为什么存在速度差异的唯一方法是查看每个查询的查询计划并查看优化器的不同之处。根据经验，其他人发现将OR子句更改为UNION子句有助于优化器更好地使用索引，从而提高查询执行时间。

这篇文章很好地解释了影响查询计划的一些变量，包括选择性。

为什么 UNION 导致更多的搜索而不是扫描是因为每个操作都需要满足特定的选择性要求才能有资格进行搜索。（选择性是被过滤的特定列的唯一性）。OR 发生在单个操作中，因此当组合每列的选择性并且超过一定百分比时，扫描被认为更有效。

您似乎认为以下代码块使一切变得不同：

where (table.fieldNameA = 'foo' or table.fieldNameA = 'bar')

由于默认情况下 UNION 对每个语句执行单独的操作，因此不会组合每列的选择性，从而使其执行查找的机会更大。现在由于 UNION 执行两个操作，它们需要使用上面的连接操作来匹配它们的结果集。通常，这不是一项昂贵的操作。

为什么我在UNION这里提到是因为您优化的 SQL 的行为与UNION.

exists (
            select * 
                from tableB, tableC, tableD
                where (table.fieldNameA = 'foo') and
                   /* More clauses */
        ) or 
        exists (
            select * 
                from tableB, tableC, tableD
                where (table.fieldNameA = 'bar') and
                    /* More clauses */
        )

您有两个单独的子查询，每个子查询都可以并行化，然后在完成时连接在一起。这些子查询中的每一个也具有更好的选择性比，鼓励使用查找而不是扫描。

您的（更新的）查询计划支持这一点。在你的第一个计划中，在底部，你有这个：

Concatenation
        |--Nested Loops(Inner Join)
        |    |--Clustered Index Seek
        |    |--Nested Loops
        |         |--Index Seek
        |         |--Clustered Index Seek
        |--Nested Loops(Inner Join)
             |--Clustered Index Seek
             |--Nested Loops
                  |--Index Seek
                  |--Clustered Index Seek

表明OR条件已被分成两个可以并行化的独立作业，随着选择性的提高，索引搜索很可能表现得更好。

每当您有一个性能不佳的查询时，有必要查看查询计划以查看哪些子部分花费了很长时间，并首先尝试修复这些子部分。非常类似于您如何优化程序的性能。

查询很难通过肉眼优化，因为很多变量会影响查询计划，例如表中的行数、可用索引和某些列的选择性。

sql - 为什么我的分解后的 SQL 运行速度要快几个数量级？

1 回答 1

Related

Reference