3

(这都是 Oracle 10g):

CREATE OR REPLACE FUNCTION bar(...)
IS
    v_first_type VARCHAR2(100) ;
    v_second_type VARCHAR2(100);

    CURSOR cur IS SELECT a,b FROM source_table ;
    v_a int;
    v_b char;
BEGIN
    OPEN cur;
    <<l_next>> --10G doesn't have the continue statement.
    LOOP
        FETCH cur INTO v_a, v_b ;
        EXIT WHEN cur%NOTFOUND ;

        --Ignore Record Case: ignore the record entirely
        IF a == -1 THEN
            -- do something
            GOTO l_next ; --10g doesn't have the continue statement.
        ELSE
            -- do something else
            v_first := 'SUCCESS' ;
        END IF;

        -- Transform Case:
        IF b == 'z' THEN
            -- do something
            v_second := 'something';
        ELSE
            -- do something
            v_second := 'something else';
        END IF;


        INSERT INTO report_table VALUES (v_first, v_second);
    END LOOP;
    CLOSE cur;
EXCEPTION 
    ...
END;

我是大学毕业后的第一份工作。我正在查看一些看起来像上面的通用框架的遗留代码(除了它有几百行长并且使用更复杂的处理(不可能有基于集合的解决方案))。

它将许多行从一个表中拉到一个游标中,遍历游标,对其进行转换,并将结果插入到报表中。游标不会插入每条记录——如果记录有问题或者它出于某种原因不喜欢它,它将跳过记录而不插入它(参见 GOTO 语句)。

问题 1:插入在循环内一一发生,而不是在循环外的末尾执行 FORALL。

问题2:游标不使用BULK COLLECT。

最重要的是,有一个存储过程有一个游标,它再次不使用 BULK COLLECT,它在遍历游标中的记录时发出这个函数。在循环的每条记录的末尾发出一个提交。我在这里写的函数中没有提交。

我想重写代码看起来像这样:

CREATE OR REPLACE FUNCTION bar(...)
IS
    CURSOR cur IS SELECT a,b FROM source_table ;

    TYPE t_source IS TABLE OF cur%ROWTYPE INDEX BY PLS_INTEGER;
    TYPE t_report IS TABLE OF destination_table%ROWTYPE INDEX BY PLS_INTEGER;
    v_sources t_source;
    v_reports t_report
    v_report_inx INT := 0; -- To Prevent Sparse Collection
BEGIN
    OPEN cur;
    <<l_next>> --10G doesn't have the continue statement.
    LOOP
        FETCH cur BULK COLLECT INTO v_sources LIMIT 100 ;
        EXIT WHEN v_sources.count = 0 ;

        FOR i IN 1 .. v_sources LOOP
            --Ignore Record Case: ignore the record entirely
            IF v_sources(i).a == -1 THEN
                -- do something
                GOTO l_next ; --10g doesn't have the continue statement.
            ELSE
                -- do something else
                v_reports(v_report_inx).first := 'SUCCESS' ;
            END IF;

            -- Transform Case:
            IF v_sources(i).b == 'z' THEN
                -- do something
                v_reports(v_report_inx).second := 'something';
            ELSE
                -- do something
                v_reports(v_report_inx).second := 'something else';
            END IF;

            v_report_inx := v_report_inx + 1;
        END LOOP;


    END LOOP;

    FORALL i in 1 .. v_reports.count
            INSERT INTO report_table (first, second) VALUES (v_reports(i).first, v_reports(i).v_second);

    CLOSE cur;
EXCEPTION 
    ...
END;

重大变化是 1) 使用 BULK COLLECT 到关联数组中,以及 2) 使用 FORALL 出不同的关联数组。

我有两个问题:

1)基于我在第一个片段中提供的框架,我的更改是最优越的方法吗?你会用另一种方式吗?

2) 有什么我没有想到的理由会让人不使用 BULK COLLECT 和 FORALL?也许一些我还没有在遗留代码中实现的复杂处理?该代码最初是在 2002 年形成的(所以我假设是 8i 或 9i),但此后一直在更新。9i 有批量装订8i 也有批量装订。两者都有关联数组。所以我觉得他们没有使用批量绑定一定是有原因的。

4

2 回答 2

2

迁移到bulk collect. 批量操作只是最小化了上下文切换和数据库往返的次数。

您的代码只有一个通用的问题。LIMIT子句防止批量操作过度使用内存,因此将它与批量收集一起使用是正确的决定。却v_reports变得不受控制。因此将批量插入移动到循环内并v_reports随后清除。

修改后的代码存在一些不准确之处。请查看下面的代码片段,/**/风格的评论是我的。

CREATE OR REPLACE FUNCTION bar(...)
IS
    CURSOR cur IS SELECT a,b FROM source_table ;

    TYPE t_source IS TABLE OF cur%ROWTYPE INDEX BY PLS_INTEGER;
    TYPE t_report IS TABLE OF destination_table%ROWTYPE INDEX BY PLS_INTEGER;
    v_sources t_source;
    v_reports t_report

    /* 1. correct type is same as type of index
       2. There are nothing wrong with sparse collections, but a separate 
           counter which incremented continuously needed for t_report.
    */
    v_report_inx PLS_INTEGER := 0; -- To Prevent Sparse Collection

BEGIN
    OPEN cur;
    <<l_next>> --10G doesn't have the continue statement.
    LOOP
        FETCH cur BULK COLLECT INTO v_sources LIMIT 100 ;

        /* On last step v_sources.count < 100, not exactly 0.
           Also if there are no elements then no processing done, 
           so check at the end of loop.
        EXIT WHEN v_sources.count = 0;
        */

        /* correct way is to loop from 1 to count
          (.last and .first not usable because both is null for empty array)
        */
        FOR i IN 1  .. v_sources.count LOOP

            v_report_inx := v_report_inx + 1;

            --Ignore Record Case: ignore the record entirely
            IF v_sources(i).a = -1 THEN
                -- do something
                GOTO l_next ; --10g doesn't have the continue statement.
            END IF;

            /* No need for ELSE here, just execution continues */
             -- do something else
             v_reports(v_report_inx).first := 'SUCCESS' ;


            -- Transform Case:
            IF v_sources(i).b = 'z' THEN
                -- do something
                v_reports(v_report_inx).second := 'something';
            ELSE
                -- do something
                v_reports(v_report_inx).second := 'something else';
            END IF;

        END LOOP;


        /* Use "indicies of" construct to deal with sparsed collections */
        FORALL i in indices of v_reports
              /* t_report already declared with %ROWTYPE 
                 so just insert entire row, it works faster */
              INSERT INTO report_table VALUES v_reports(i);

        /* Cleanup after insert */
        v_reports.delete;

        /* If number of selected records less than LIMIT then last row reached. */
        EXIT WHEN v_sources.count < 100;

    END LOOP;


    CLOSE cur;
EXCEPTION
    ...
END;

更新

感谢@jonearles。他鼓励我测试在 PL/SQL 中处理游标的不同方法的性能。

以下是 3 000 000 条记录的测试结果。很明显,从普通显式游标迁移到批量收集方法会带来真正的性能提升。
同时具有批量收集选项的显式游标和正确选择的 LIMIT 总是优于隐式游标,但它们之间的区别在于可接受的范围。

Variant name           | Time (sec)
-------------------------------------
bulk_cursor_limit_500  |  1.26
bulk_cursor_limit_100  |  1.52
bulk_unlimited         |  1.75
implicit_cursor        |  1.83
plain_cursor           | 27.20

下面是测试代码(这里是有限的 SQLFiddle 示例)

方案设置

drop table t
/
drop table log_run
/
create table t(a number, b number)
/
insert into t select level, level from dual connect by level <= 3000000
/

create table log_run(id varchar2(30), seconds number);
/

delete log_run
/

单次试运行

declare
  cursor test_cur is
    select a, b from t;

  test_rec test_cur%rowtype;
  counter    number;

  vStart timestamp;
  vEnd timestamp;
  vTimeFormat varchar2(30) := 'SSSSS.FF9';
begin

  vStart := systimestamp;

  open test_cur;
  loop
    fetch test_cur into test_rec;
    exit when test_cur%notfound;
    counter := counter + 1;
  end loop;
  close test_cur;

  vEnd := systimestamp;
  insert into log_run(id, seconds) 
    values('plain_cursor', 
             to_number(to_char(vEnd,vTimeFormat))
             -
             to_number(to_char(vStart,vTimeFormat)) 
          )
  ;

end;
/

--Implicit cursor
--0.2 seconds
declare
  test_rec   t%rowtype;
  counter    number;

  vStart timestamp;
  vEnd timestamp;
  vTimeFormat varchar2(30) := 'SSSSS.FF9';
begin

  vStart := systimestamp;

  for c_test_rec in (select a, b from t) loop
    test_rec.a := c_test_rec.a;
    test_rec.b := c_test_rec.b;
    counter := counter + 1;
  end loop;

  vEnd := systimestamp;
  insert into log_run(id, seconds) 
    values('implicit_cursor', 
             to_number(to_char(vEnd,vTimeFormat))
             -
             to_number(to_char(vStart,vTimeFormat)) 
          )
  ;

end;
/

declare
  cursor test_cur is
    select a, b from t;

  type t_test_table is table of t%rowtype;

  test_tab   t_test_table;
  counter    number;

  vStart timestamp;
  vEnd timestamp;
  vTimeFormat varchar2(30) := 'SSSSS.FF9';
begin

  vStart := systimestamp;

  open test_cur;
  loop
    fetch test_cur bulk collect into test_tab limit 100;
    for i in 1 .. test_tab.count loop
      counter := counter + 1;
    end loop;

    exit when test_tab.count < 100;
  end loop;

  close test_cur;

  vEnd := systimestamp;
  insert into log_run(id, seconds) 
    values('bulk_cursor_limit_100', 
             to_number(to_char(vEnd,vTimeFormat))
             -
             to_number(to_char(vStart,vTimeFormat)) 
          )
  ;

end;
/


declare
  cursor test_cur is
    select a, b from t;

  type t_test_table is table of t%rowtype;

  test_tab   t_test_table;
  counter    number;

  vStart timestamp;
  vEnd timestamp;
  vTimeFormat varchar2(30) := 'SSSSS.FF9';
begin

  vStart := systimestamp;

  open test_cur;
  loop
    fetch test_cur bulk collect into test_tab limit 500;
    for i in 1 .. test_tab.count loop
      counter := counter + 1;
    end loop;

    exit when test_tab.count < 500;
  end loop;

  close test_cur;

  vEnd := systimestamp;
  insert into log_run(id, seconds) 
    values('bulk_cursor_limit_500', 
             to_number(to_char(vEnd,vTimeFormat))
             -
             to_number(to_char(vStart,vTimeFormat)) 
          )
  ;

end;
/

declare

  type t_test_table is table of t%rowtype;

  test_tab   t_test_table;
  counter    number;

  vStart timestamp;
  vEnd timestamp;
  vTimeFormat varchar2(30) := 'SSSSS.FF9';
begin

  vStart := systimestamp;

  select * bulk collect into test_tab from t;

  for i in 1 .. test_tab.count loop
    counter := counter + 1;
  end loop;

  vEnd := systimestamp;
  insert into log_run(id, seconds) 
    values('bulk_unlimited', 
             to_number(to_char(vEnd,vTimeFormat))
             -
             to_number(to_char(vStart,vTimeFormat)) 
          )
  ;

end;
/

选择平均结果

select * from ( 
  select lr.id, trunc(avg(seconds),2) seconds  
  from log_run lr group by lr.id) 
  order by seconds
)
于 2013-08-10T09:11:09.483 回答
1

我会重写它以便不使用 GOTO(我想我只是一个老的未重构的结构化程序员 :-)。我也会摆脱显式游标并使用游标 FOR 循环,从 10g 开始通常会在幕后批量绑定。尝试:

CREATE OR REPLACE FUNCTION bar(...)
IS
    v_first_type VARCHAR2(100) ;
    v_second_type VARCHAR2(100);
BEGIN
    <<OUTER_LOOP>>
    FOR aRow In (SELECT A, B FROM SOURCE_TABLE)
    LOOP

      <<INNER_LOOP>>
      LOOP  -- This loop is used to allow us to skip the later INSERT, and
            -- will only be passed through once for each row returned by
            -- the FOR loop.
        --Ignore Record Case: ignore the record entirely
        IF aRow.A == -1 THEN
            -- do something
            EXIT INNER_LOOP;  -- rather than GOTO
        ELSE
            -- do something else
            v_first := 'SUCCESS' ;
        END IF;

        -- Transform Case:
        IF aRow.B == 'z' THEN
            -- do something
            v_second := 'something';
        ELSE
            -- do something
            v_second := 'something else';
        END IF;


        INSERT INTO report_table VALUES (v_first, v_second);

        EXIT INNER_LOOP;  -- the "loop" is used to allow the INSERT to be
                          -- skipped and thus we don't ever want to go back
                          -- to the top
      END LOOP;  -- INNER_LOOP
    END LOOP;  -- OUTER_LOOP
EXCEPTION 
    ...
END;

请注意使用带有显式退出的内部循环来保持自上而下的流程,同时允许显式控制循环。

我还建议在 Oracle 分析器下运行此代码,以了解哪些代码行占用的时间最多。试图通过猜测瓶颈在哪里来优化代码是浪费时间。直到你对它进行分析,你才会猜到——而且你永远猜不到我猜错的频率。:-) 代码花时间在最该死的地方...

分享和享受。

于 2013-08-10T11:21:30.670 回答