1

比下面更简单地说:如果一个有一个或多个查询参数,例如x_id,(或报告/表函数参数),这些参数对性能至关重要(例如可以使用一些主键索引)并且它可能是(取决于用例/report 已应用过滤器,...) 其中之一

  • null
  • 完全匹配(例如一些唯一的id)
  • 类似的表达
  • 甚至是正则表达式

那么如果所有这些可能性都编码在一个查询中,我只会看到并知道优化器

  • 生成唯一的静态计划,独立于实际参数runtime-value
  • 因此不能假设使用一些索引x_id尽管它可能是例如一些完全匹配

除了_ _

  1. 让一些 PL/SQL 代码从n预定义和用例优化的查询/视图中进行选择?
    • 一个人拥有的灵活参数越多,它就越大
  2. 还是一些手动字符串构造和动态编译的查询?

基本上,我有两个略有不同的用例/问题,如下所示和可执行文件:

一个-select * from tf_sel

-select * from data_union

这可能会通过SQL 提示或使用其他技巧来解决。

为了加快这些查询的速度,我目前在某个实现级别(表函数)上分离“合并查询” ,这非常麻烦且难以维护,但确保查询由于其更好的执行计划而运行得非常快。

正如我所看到的,主要问题似乎是优化器 sql 计划的静态性质,如果它考虑一些“查询时间常数”过滤器参数,它可能会更有效,但它总是相同的。

with
    -- Question A: What would be a good strategy to make tf_sel with tf_params nearly as fast as query_use_case_1_eq
    --             which actually provides the same result?
    --
    -- - a complex query should be used in various reports with filters
    -- - we want to keep as much as possible filter functionality on the db side (not the report engine side)
    --   to be able to utilize the fast and efficient db engine and for loosely coupled software design


    complex_query as (  -- just some imaginable complex query with a lot of table/view joins, aggregation/analytical functions etc.
        select 1 as id, 'ab12' as indexed_val, 'asdfasdf' x from dual
        union all select 2, 'ab34', 'a uiop345' from dual
        union all select 3, 'xy34', 'asdf  0u0duaf' from dual
        union all select 4, 'xy55', ' asdja´sf asd' from dual
    )


-- <<< comment the following lines in to test it with the above

--  , query_use_case_1_eq as (  -- quite fast and maybe the 95% use case
--      select * from complex_query where indexed_val = 'ab12'
--  )
--select * from query_use_case_1_eq 

-- >>>

-- ID INDEXED_VAL X
-- -- ----------- --------
--  1 ab12        asdfasdf


-- <<< comment the following lines in to test it with the above

--  , query_use_case_2_all as (  -- significantly slower due to a lot of underlying calculations
--      select * from complex_query
--  )
--select * from query_use_case_2_all

-- >>>

-- ID INDEXED_VAL X
-- -- ----------- -------------
--  1 ab12        asdfasdf
--  2 ab34        a uiop345
--  3 xy34        asdf  0u0duaf
--  4 xy55         asdja´sf asd


-- <<< comment the following lines in to test it with the above

--  , query_use_case_3_like as (
--      select * from complex_query where indexed_val like 'ab%'
--  )
--select * from query_use_case_3_like

-- >>>

-- ID INDEXED_VAL X
-- -- ----------- ---------
--  1 ab12        asdfasdf
--  2 ab34        a uiop345


-- <<< comment the following lines to simulate the table function

    , tf_params as (  -- table function params: imagine we have a table function where these are passed depending on the report
        select  'ab12' p_indexed_val,  'eq' p_filter_type  from dual
    )
    , tf_sel as (  -- table function select: nicely integrating all query possiblities, but beeing veeery slow :-(
        select q.* 
        from 
            tf_params p  -- just here so this example works without the need for the actual function
            join complex_query q on (1=1)
        where
                p_filter_type = 'all'
            or (p_filter_type = 'eq' and indexed_val = p_indexed_val)
            or (p_filter_type = 'like' and indexed_val like p_indexed_val)
            or (p_filter_type = 'regexp' and regexp_like(indexed_val, p_indexed_val))
    )

-- actually we would pass the tf_params above if it were a real table function
select * from tf_sel

-- >>>

-- ID INDEXED_VAL X
-- -- ----------- --------
--  1 ab12        asdfasdf



-- Question B: How can we speed up data_union with dg_filter to be as fast as the data_group1 query which
--             actually provides the same result? 
-- 
-- A very similar approach is considered in other scenarios where we like to join the results of 
-- different queries (>5) returning joinable data and beeing filtered based on the same parameters.

-- <<< comment the following lines to simulate the union problem

--  , data_group1 as (  -- may run quite fast
--      select 'dg1' dg_id, q.* from complex_query q where x < 'a'  -- just an example returning some special rows that should be filtered later on!
--  )
--  
--  , data_group2 as (  -- may run quite fast
--      select 'dg2' dg_id, q.* from complex_query q where instr(x,'p') >= 0  -- just an example returning some special rows that should be filtered later on!
--  )   
--  
--  
--  , dg_filter as (  -- may be set by a report or indirectly by user filters
--      select  'dg1' dg_id  from dual
--  )
--  
--  , data_union as (  -- runs much slower due to another execution plan
--      select * from (
--          select * from data_group1 
--          union all select * from data_group2
--      )
--      where dg_id in (select dg_id from dg_filter)
--  )
--
--select * from data_union

-- >>>

-- DG_ID ID INDEXED_VAL X
-- ----- -- ----------- -------------
-- dg1    4 xy55         asdja´sf asd

这是对 jonearles 提供的示例代码和答案的评论

实际上,您的答案是我的(尽管在某些情况下一起出现但不相关)用例 A 和 B 的混合。尽管您提到优化器具有动态FILTER功能和其他功能仍然很重要。

用例 B(“数据分区/组联合”)

实际上用例 B(基于您的示例表)看起来更像这样,但我仍然需要检查实际场景中的性能问题。也许您已经看到了一些问题?

select * from (
    select 'dg1' data_group, x.* from sample_table x 
        where mod(to_number(some_other_column1), 100000) = 0  -- just some example restriction
            --and indexed_val = '3635'  -- commenting this in and executing this standalone returns:
            ----------------------------------------------------------------------------------------
            --| Id  | Operation                   | Name              | Rows  | Bytes | Cost (%CPU)|
            ----------------------------------------------------------------------------------------
            --|   0 | SELECT STATEMENT            |                   |     1 |    23 |     2   (0)|
            --|   1 |  TABLE ACCESS BY INDEX ROWID| SAMPLE_TABLE      |     1 |    23 |     2   (0)|
            --|   2 |   INDEX RANGE SCAN          | SAMPLE_TABLE_IDX1 |     1 |       |     1   (0)|
            ----------------------------------------------------------------------------------------            
    union all
    select 'dg2', x.* from sample_table x
        where mod(to_number(some_other_column2), 9999) = 0  -- just some example restriction
    union all
    select 'dg3', x.* from sample_table x
        where mod(to_number(some_other_column3), 3635) = 0  -- just some example restriction
)
where data_group in ('dg1') and indexed_val = '35'

-------------------------------------------------------------------------------------------
--| Id  | Operation                      | Name              | Rows  | Bytes | Cost (%CPU)|
-------------------------------------------------------------------------------------------
--|   0 | SELECT STATEMENT               |                   |     3 |   639 |     2   (0)|
--|   1 |  VIEW                          |                   |     3 |   639 |     2   (0)|
--|   2 |   UNION-ALL                    |                   |       |       |            |
--|   3 |    TABLE ACCESS BY INDEX ROWID | SAMPLE_TABLE      |     1 |    23 |     2   (0)|
--|   4 |     INDEX RANGE SCAN           | SAMPLE_TABLE_IDX1 |     1 |       |     1   (0)|
--|   5 |    FILTER                      |                   |       |       |            |
--|   6 |     TABLE ACCESS BY INDEX ROWID| SAMPLE_TABLE      |     1 |    23 |     2   (0)|
--|   7 |      INDEX RANGE SCAN          | SAMPLE_TABLE_IDX1 |     1 |       |     1   (0)|
--|   8 |    FILTER                      |                   |       |       |            |
--|   9 |     TABLE ACCESS BY INDEX ROWID| SAMPLE_TABLE      |     1 |    23 |     2   (0)|
--|  10 |      INDEX RANGE SCAN          | SAMPLE_TABLE_IDX1 |     1 |       |     1   (0)|
-------------------------------------------------------------------------------------------

用例 A(按列查询类型过滤)

根据您的示例表,这更像是我想做的。正如您所看到的,只有 fast 的查询where p.ft_id = 'eq' and x.indexed_val = p.val显示了索引使用情况,但是在where子句中使用所有不同的过滤器选项将导致计划切换始终使用全表扫描:-/:p_filter_type( 即使我:p_indexed_val_filter在SQL 不仅仅是我把它放在一个地方,它不会改变。)

with 
    filter_type as (
        select 'all' as id from dual
        union all select 'eq' as id from dual
        union all select 'like' as id from dual
        union all select 'regexp' as id from dual
    )
    , params as (
        select 
            (select * from filter_type where id = :p_filter_type) as ft_id,
            :p_indexed_val_filter as val
        from dual
    )
select * 
from params p
    join sample_table x on (1=1)
    -- the following with the above would show the 'eq' use case with a fast index scan (plan id 14/15)
    --where p.ft_id = 'eq' and x.indexed_val = p.val
    ------------------------------------------------------------------------------------------
    --| Id  | Operation                     | Name              | Rows  | Bytes | Cost (%CPU)|
    ------------------------------------------------------------------------------------------
    --|   0 | SELECT STATEMENT              |                   |     1 |    23 |    12   (0)|
    --|   1 |  VIEW                         |                   |     4 |    20 |     8   (0)|
    --|   2 |   UNION-ALL                   |                   |       |       |            |
    --|   3 |    FILTER                     |                   |       |       |            |
    --|   4 |     FAST DUAL                 |                   |     1 |       |     2   (0)|
    --|   5 |    FILTER                     |                   |       |       |            |
    --|   6 |     FAST DUAL                 |                   |     1 |       |     2   (0)|
    --|   7 |    FILTER                     |                   |       |       |            |
    --|   8 |     FAST DUAL                 |                   |     1 |       |     2   (0)|
    --|   9 |    FILTER                     |                   |       |       |            |
    --|  10 |     FAST DUAL                 |                   |     1 |       |     2   (0)|
    --|  11 |  FILTER                       |                   |       |       |            |
    --|  12 |   NESTED LOOPS                |                   |     1 |    23 |     4   (0)|
    --|  13 |    FAST DUAL                  |                   |     1 |       |     2   (0)|
    --|  14 |    TABLE ACCESS BY INDEX ROWID| SAMPLE_TABLE      |     1 |    23 |     2   (0)|
    --|  15 |     INDEX RANGE SCAN          | SAMPLE_TABLE_IDX1 |     1 |       |     1   (0)|
    --|  16 |   VIEW                        |                   |     4 |    20 |     8   (0)|
    --|  17 |    UNION-ALL                  |                   |       |       |            |
    --|  18 |     FILTER                    |                   |       |       |            |
    --|  19 |      FAST DUAL                |                   |     1 |       |     2   (0)|
    --|  20 |     FILTER                    |                   |       |       |            |
    --|  21 |      FAST DUAL                |                   |     1 |       |     2   (0)|
    --|  22 |     FILTER                    |                   |       |       |            |
    --|  23 |      FAST DUAL                |                   |     1 |       |     2   (0)|
    --|  24 |     FILTER                    |                   |       |       |            |
    --|  25 |      FAST DUAL                |                   |     1 |       |     2   (0)|
    ------------------------------------------------------------------------------------------  
where 
    --mod(to_number(some_other_column1), 3000) = 0 and  -- just some example restriction
    (
            p.ft_id = 'all'
        or
            p.ft_id = 'eq' and x.indexed_val = p.val
        or 
            p.ft_id = 'like' and x.indexed_val like p.val
        or 
            p.ft_id = 'regexp' and regexp_like(x.indexed_val, p.val)
    )
-- with the full flexibility of the filter the plan shows a full table scan (plan id 13) :-(    
--------------------------------------------------------------------------
--| Id  | Operation          | Name         | Rows  | Bytes | Cost (%CPU)|
--------------------------------------------------------------------------
--|   0 | SELECT STATEMENT   |              |  1099 | 25277 |   115   (3)|
--|   1 |  VIEW              |              |     4 |    20 |     8   (0)|
--|   2 |   UNION-ALL        |              |       |       |            |
--|   3 |    FILTER          |              |       |       |            |
--|   4 |     FAST DUAL      |              |     1 |       |     2   (0)|
--|   5 |    FILTER          |              |       |       |            |
--|   6 |     FAST DUAL      |              |     1 |       |     2   (0)|
--|   7 |    FILTER          |              |       |       |            |
--|   8 |     FAST DUAL      |              |     1 |       |     2   (0)|
--|   9 |    FILTER          |              |       |       |            |
--|  10 |     FAST DUAL      |              |     1 |       |     2   (0)|
--|  11 |  NESTED LOOPS      |              |  1099 | 25277 |   115   (3)|
--|  12 |   FAST DUAL        |              |     1 |       |     2   (0)|
--|  13 |   TABLE ACCESS FULL| SAMPLE_TABLE |  1099 | 25277 |   113   (3)|
--|  14 |    VIEW            |              |     4 |    20 |     8   (0)|
--|  15 |     UNION-ALL      |              |       |       |            |
--|  16 |      FILTER        |              |       |       |            |
--|  17 |       FAST DUAL    |              |     1 |       |     2   (0)|
--|  18 |      FILTER        |              |       |       |            |
--|  19 |       FAST DUAL    |              |     1 |       |     2   (0)|
--|  20 |      FILTER        |              |       |       |            |
--|  21 |       FAST DUAL    |              |     1 |       |     2   (0)|
--|  22 |      FILTER        |              |       |       |            |
--|  23 |       FAST DUAL    |              |     1 |       |     2   (0)|
--------------------------------------------------------------------------
4

2 回答 2

1

一些特性使优化器能够生成动态计划。最常见的功能是FILTER操作,不应与过滤谓词混淆。操作允许 Oracle在FILTER运行时根据动态值启用或禁用部分计划。此功能通常适用于绑定变量,其他类型的动态查询可能不会使用它。

示例架构

create table sample_table
(
    indexed_val        varchar2(100),
    some_other_column1 varchar2(100),
    some_other_column2 varchar2(100),
    some_other_column3 varchar2(100)
);

insert into sample_table
select level, level, level, level
from dual
connect by level <= 100000;

create index sample_table_idx1 on sample_table(indexed_val);

begin
    dbms_stats.gather_table_stats(user, 'sample_table');
end;
/

使用绑定变量的示例查询

explain plan for
select * from sample_table where :p_filter_type = 'all'
union all
select * from sample_table where :p_filter_type = 'eq'     and indexed_val = :p_indexed_val
union all
select * from sample_table where :p_filter_type = 'like'   and indexed_val like :p_indexed_val
union all
select * from sample_table where :p_filter_type = 'regexp' and regexp_like(indexed_val, :p_indexed_val);

select * from table(dbms_xplan.display(format => '-cost -bytes -rows'));

样品计划

这表明根据输入使用的计划有很大不同。单个=将使用INDEX RANGE SCAN,没有谓词将使用TABLE ACCESS FULL。正则表达式还使用全表扫描,因为无法索引正则表达式。尽管取决于表达式的确切类型,但可以通过基于函数的索引或 Oracle Text 索引启用有用的索引。

Plan hash value: 100704550

------------------------------------------------------------------------------
| Id  | Operation                             | Name              | Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                      |                   | 00:00:01 |
|   1 |  UNION-ALL                            |                   |          |
|*  2 |   FILTER                              |                   |          |
|   3 |    TABLE ACCESS FULL                  | SAMPLE_TABLE      | 00:00:01 |
|*  4 |   FILTER                              |                   |          |
|   5 |    TABLE ACCESS BY INDEX ROWID BATCHED| SAMPLE_TABLE      | 00:00:01 |
|*  6 |     INDEX RANGE SCAN                  | SAMPLE_TABLE_IDX1 | 00:00:01 |
|*  7 |   FILTER                              |                   |          |
|   8 |    TABLE ACCESS BY INDEX ROWID BATCHED| SAMPLE_TABLE      | 00:00:01 |
|*  9 |     INDEX RANGE SCAN                  | SAMPLE_TABLE_IDX1 | 00:00:01 |
|* 10 |   FILTER                              |                   |          |
|* 11 |    TABLE ACCESS FULL                  | SAMPLE_TABLE      | 00:00:01 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter(:P_FILTER_TYPE='all')
   4 - filter(:P_FILTER_TYPE='eq')
   6 - access("INDEXED_VAL"=:P_INDEXED_VAL)
   7 - filter(:P_FILTER_TYPE='like')
   9 - access("INDEXED_VAL" LIKE :P_INDEXED_VAL)
       filter("INDEXED_VAL" LIKE :P_INDEXED_VAL)
  10 - filter(:P_FILTER_TYPE='regexp')
  11 - filter( REGEXP_LIKE ("INDEXED_VAL",:P_INDEXED_VAL))
于 2014-03-14T04:44:45.153 回答
0

(更适用于情况A)但也适用于B)以这种方式......)

我现在正在使用一些混合方法(我的问题中的 1. 和 2. 点的组合)并且实际上非常喜欢它,因为它还提供了良好的调试和封装可能性,并且优化器根本不需要处理找到最好的基于更大查询中基本逻辑分离的查询的策略,例如内部FILTER规则,这可能很好,或者在最坏的情况下效率低得令人难以置信:

  1. 在报告中使用它

    select * 
    from table(my_report_data_func_sql(
      :val1,
      :val1_filter_type,
      :val2
    ))
    
  2. 表函数是这样定义的

    create or replace function my_report_data_func_sql(
      p_val1              integer   default 1234,
      p_val1_filter_type  varchar2  default 'eq',
      p_val2              varchar2  default null
    ) return varchar2 is
    
      query varchar2(4000) := ' 
        with params as (  -- *: default param
            select
                ''||p_val1||'' p_val1,                            -- eq*
                '''||p_val1_filter_type||''' p_val1_filter_type,  -- [eq, all*, like, regexp]
                '''||p_val2||''' p_val2                           -- null*
            from dual
        )
        select x.*
        from
            params p  -- workaround for standalone-sql-debugging using "with" statement above
            join my_report_data_base_view x on (1=1)
        where 1=1  -- ease of filter expression adding below
        '          
        -- #### FILTER CRITERIAS are appended here ####
    
        -- val1-filter
        ||case p_val1_filter_type
            when 'eq' then '
            and val1 = p_val1
        '   when 'like' then '
            and val1 like p_val1
        '   when 'regexp' then '
            and regexp_like(val1,  p_val1)
        '   else '' end  -- all
    
      ;
    begin
      return query;
    end;
    ;
    
  3. 并将通过示例生成以下内容:

    select * 
    from table(my_report_data_func_sql(
      1234,
      'eq',
      'someval2'
    ))
    
    /*
            with params as (  -- *: default param
                select
                    1          p_val1,              -- eq*
                    'eq'       p_val1_filter_type,  -- [eq, all*, like, regexp]
                    'someval2' p_val2               -- null*
                from dual
            )
            select x.*
            from
                params p  -- workaround for standalone-sql-debugging using "with" statement above
                join my_report_data_base_view x on (1=1)
            where 1=1  -- ease of filter expression adding below
                and val1 = p_val1
    */
    
于 2014-04-04T07:08:39.730 回答