2

我有以下场景:一个最多包含 20000 个条目的表和另一个具有相应自定义字段的表。我需要在所有列(包括自定义字段)上实现一个带有过滤机会的查询,并跳过并获取,并且我需要过滤后的总行数。在动态 sql 的帮助下,我设法实现了一个查询,它将自定义字段作为列添加到第一个表中。但是我真的很难实现运行非常快并且还返回总行数的skip and take功能。由于并非我们所有的客户都在 SQL Server 2012 上运行,因此最好的方法是通过 row_number 上的 where 子句实现 take 和 skip,但我认为这也是最慢的选择。我喜欢 2012 案例的 OFFSET 和 FETCH 功能,

这是我从动态 sql 得到的查询,在整个动态查询下方,都有两个备选方案作为注释

With tempTable AS 
(
SELECT *
 --1. ALTERNATIVE:
,ROW_NUMBER() Over (ORDER BY  Nachname, Vorname desc) As  ROW_NUMBER ,COUNT(1) OVER () as Total_Rows
FROM
(
    SELECT  [a].*, [DFSF].[Datenfeld_Name],[Datenfeld_Inhalt]
    FROM 
    [dbo].[Ansprechpartner] AS a left join [dbo].[Datenfeld] AS dfe
    ON [a].[Id] = [dfe].[Datenfeld_AnsprechpartnerID]
    left join
    [Datenfeld_Standardfelder] AS DFSF 
    ON dfe.[StandardfeldID] = [DFSF].[id] and datenfeld_kategorie = 'Ansprechpartner'   

) AS j
PIVOT
(
  max([Datenfeld_Inhalt]) FOR [j].[Datenfeld_Name]  IN ([Medium],[Kontaktthema],[Mediengattung],[Medienthema],[E-Mail],[Homepage],[Rolle])
) AS p
)
SELECT *
--2. ALTERNATIVE:
--,COUNT(1) OVER ()
FROM tempTable
 WHERE 1=1
 -- 1. ALTERNATIVE:
and Row_Number BETWEEN 0 AND 100
ORDER BY  Nachname, Vorname DESC
--2. ALTERNATIVE:
OFFSET 0 ROWS FETCH NEXT 100 ROWS ONLY
 ; 

并遵循整个动态查询。我实际上认为,其中有一个错误,因为这样我不会得到正确的行数,我可能不得不在没有 row_number 过滤器的情况下再次调用它以使其正确......

DECLARE @filterExpression nvarchar(MAX)
DECLARE @showOnlyDoublets int
DECLARE @sortExpression nvarchar(MAX)
 DECLARE @skip AS int
 DECLARE @take AS [int]
 SELECT @skip = 0
 SELECT @take = 100
--SELECT @filterExpression = 'WHERE Vorname like ''%luc%'''
SELECT @filterExpression = ' WHERE 1=1'
SELECT @sortExpression = 'ORDER BY  Nachname, Vorname desc'
SELECT @showOnlyDoublets = 0

DECLARE @idList nvarchar(MAX)
select @idList =  COALESCE(@idList + '],[', '[') + [DFSF].[Datenfeld_Name] from
[Datenfeld_Standardfelder] AS DFSF 
where datenfeld_kategorie = 'Ansprechpartner'

SELECT @idList = @idList +']'
--SELECT @idList

DECLARE @sqlToRun nvarchar(max)
SET @sqlToRun = 
'With tempTable As 
(
SELECT *
, ROW_NUMBER() Over (' + @sortExpression + ') As  Row_Number 
FROM
(
    SELECT  [a].*, [DFSF].[Datenfeld_Name],[Datenfeld_Inhalt]--, CAST( ROW_NUMBER()  OVER(ORDER BY [DFSF].[Datenfeld_Name] DESC) AS varchar(20)) 
    FROM 
    [dbo].[Ansprechpartner] AS a left join [dbo].[Datenfeld] AS dfe
    ON [a].[Id] = [dfe].[Datenfeld_AnsprechpartnerID]
    left join
    [Datenfeld_Standardfelder] AS DFSF 
    ON dfe.[StandardfeldID] = [DFSF].[id] and datenfeld_kategorie = ''Ansprechpartner'' 

) AS j
PIVOT
(
  max([Datenfeld_Inhalt]) FOR [j].[Datenfeld_Name]  IN (' + @idList + ')
) AS p
)
SELECT *,  COUNT(*) OVER () as Total_Rows FROM tempTable
' + @filterExpression +  '
AND Row_Number BETWEEN ' + CAST ( @skip AS varchar ) + ' AND ' + CAST ( @take AS varchar ) + '
' + @sortExpression + '
--OFFSET ' + CAST ( @skip AS varchar ) + ' ROWS FETCH NEXT ' + CAST ( @take AS varchar ) + ' ROWS ONLY
 ;'


PRINT @sqlToRun
EXECUTE sp_executesql @sqlToRun

所以我的问题是:有没有办法改进这个查询(两种选择之一)?还是您有完全不同的想法,因为我认为,无论哪种方式,如果我正确地调用计数,都会花费很多时间。

4

1 回答 1

1

弗里德,

看看下面这个存储过程。我添加的评论希望能有所帮助。

它提供整个查询的总行数,无论每页返回多少条记录或用户在哪个页面上。可能干扰您的行数的是行号是代码中 where 子句的一部分。在此示例中查看如何返回子集,而无需在 where 子句中处理该逻辑。

您的性能问题可能与索引和旋转有关。我看到您正在通过任意文本输入过滤列。您需要将该列与所有其他返回的列一起编入索引。下面的查询中描述了一种可能加快所有速度的替代方法,在该查询中,您首先找到与您的谓词匹配的所有 ID(在搜索列上具有非聚集索引并包括主键 ID 列),然后加入实际查询到那张桌子。这将整个操作(包括枢轴)限制为仅匹配的行。

CREATE PROC [api].[cc_pcp_member_search] 
(
     @string VARCHAR(255)    
    ,@plan_long_id BIGINT 
    ,@num_rows BIGINT
    ,@page_num BIGINT
    ,@order_by VARCHAR(255) = 'last_name'
    ,@sort_order VARCHAR(10) = 'ASC'
)

AS 
SET NOCOUNT ON

DECLARE @MemberList AS TABLE (MemberId INT)
INSERT INTO @MemberList

/* What is happening in the EXEC statement below, is that I am running a text search against
very specific indexes that only return the ID's(clustered) of the items I care about.
This has proven to be far more performant than applying the text predicate to the entire
set due to indexing reasons. The 'real' query further below that brings back all the needed fields
uses this list of ID's to define the scope of the set. You might consider something similar if 
you are querying aginst arbitrary text values.
*/
EXEC dbo.uspGetMemberIdForMemberSearch @string 

/* The table variable does slow things down and you might not need it at all.
In this case, the developers were using Entity Framework and it was having 
trouble determining the signature and data types of the stored procedure without
a typed result set
*/
DECLARE @ResultSet AS TABLE (   member_id BIGINT,
                first_name VARCHAR(255),
                last_name VARCHAR(255),
                full_name VARCHAR(255),
                hicn VARCHAR(255),
                gender VARCHAR(255),
                plan_id BIGINT,
                plan_name VARCHAR(255),
                phone_1 VARCHAR(255),
                phone_2 VARCHAR(255),
                dob DATETIME,
                total_records INT)

INSERT INTO @ResultSet
SELECT
CAST(M.MemberId AS BIGINT) AS member_id,
M.FirstName AS first_name,
M.LastName AS last_name,
CONCAT(M.FirstName, ' ', M.LastName) AS full_name,
M.HICN AS hicn,
CASE WHEN M.Gender = 0 THEN 'F' ELSE 'M' END AS gender,
CAST(P.VirtusPlanId AS BIGINT) AS plan_id,
P.PlanName AS plan_name,
M.PhoneNumber1 AS phone_1,
M.PhoneNumber2 AS phone_2,
M.DateOfBirth AS dob,
CAST(0 AS INT) AS total_records --<<< This is the place holder for total count, see further below
FROM
  Member M
  INNER JOIN [Plan] P
    ON M.PlanId = P.PlanId
  INNER JOIN @MemberList ML --<<< This is where the final filtering on ID's happens
    ON M.MemberId = ML.MemberId

/* Heres the core of what you are probably dealing with. The proc allows the caller
to specify sort order, query string, records per page, and current page in the set.
The users will usually search by name, sort by last name, and the grid then pages
through those results 10 rows (defaulted in app) at a time.
*/
DECLARE @InputPageNumber int = @page_num
DECLARE @RowsPerPage INT = @num_rows
DECLARE @RealPageNumber INT = @InputPageNumber + 1 --<<< 0 based index adjustment
DECLARE @OrderBy VARCHAR(255) = @order_by
DECLARE @SortOrder VARCHAR(4) = @sort_order


  SELECT
     member_id
    ,first_name
    ,last_name
    ,full_name
    ,hicn
    ,gender
    ,plan_id
    ,plan_name
    ,phone_1
    ,phone_2
    ,dob
    /* Here is your total row count of the set regardless of how many
    rows are being paged at the moment. Because OVER() is counting the
    filtered set in the table variable only, it is fast. It would do the same
    with a CTE. You're already using the same thing.
    */
    ,CAST(COUNT(*) OVER() AS INT) AS total_records
  FROM @ResultSet
  WHERE plan_id = @plan_long_id

    /* In this usecase, the output of this query feeds a paged web grid. Below is the 
    logic used to determine the sort column at execution time. I'm not pretending it's 
    pretty but it works.
    */
    ORDER BY CASE 
        WHEN @OrderBy = 'first_name' AND @SortOrder = 'ASC'
          THEN first_name
        WHEN @OrderBy = 'last_name' AND @SortOrder = 'ASC'
          THEN last_name
        WHEN @OrderBy = 'full_name' AND @SortOrder = 'ASC'
          THEN full_name
        WHEN @OrderBy = 'hicn' AND @SortOrder = 'ASC'
          THEN hicn
        WHEN @OrderBy = 'plan_name' AND @SortOrder = 'ASC'
          THEN plan_name
        WHEN @OrderBy = 'gender' AND @SortOrder = 'ASC'
          THEN gender
        END ASC,
        CASE 
        WHEN @OrderBy = 'first_name' AND @SortOrder = 'DESC'
          THEN first_name
        WHEN @OrderBy = 'last_name' AND @SortOrder = 'DESC'
          THEN last_name
        WHEN @OrderBy = 'full_name' AND @SortOrder = 'DESC'
          THEN full_name
        WHEN @OrderBy = 'hicn' AND @SortOrder = 'DESC'
          THEN hicn
        WHEN @OrderBy = 'plan_name' AND @SortOrder = 'DESC'
          THEN plan_name
        WHEN @OrderBy = 'gender' AND @SortOrder = 'DESC'
          THEN gender
        END DESC
  OFFSET (@RealPageNumber - 1) * @RowsPerPage ROWS  -- dynamic offset arithmetic to convert requested page number to row offset
  FETCH NEXT @RowsPerPage ROWS ONLY;                -- dynamic number of rows to display per page 
于 2015-01-29T18:21:30.733 回答