我们有一个相当简单的存储过程，它将所有数据聚合到一个大表中（然后将结果放在另一个表中）。对于一个 5M 行的表，这个过程大约需要 4 分钟 - 这是完全合理的。但是对于一个13M行的表，这个相同的过程大约需要 60 分钟。
看起来我们正在打破某种门槛，我很难找到一个简单的解决方法。手动将其重写为聚合表的一小部分的几个“线程”会导致 10-15 分钟的合理运行时间。
有没有办法在这里看到实际的瓶颈？
更新：这两个查询的查询计划当然是相同的，它们看起来像这样：
|ROOT:EMIT Operator (VA = 3)
|
| |INSERT Operator (VA = 2)
| | The update mode is direct.
| |
| | |HASH VECTOR AGGREGATE Operator (VA = 1)
| | | GROUP BY
| | | Evaluate Grouped SUM OR AVERAGE AGGREGATE.
| | | Evaluate Grouped COUNT AGGREGATE.
| | | Evaluate Grouped SUM OR AVERAGE AGGREGATE.
[---//---]
| | | Evaluate Grouped SUM OR AVERAGE AGGREGATE.
| | | Using Worktable1 for internal storage.
| | | Key Count: 10
| | |
| | | |SCAN Operator (VA = 0)
| | | | FROM TABLE
| | | | TableName
| | | | Table Scan.
| | | | Forward Scan.
| | | | Positioning at start of table.
| | | | Using I/O Size 16 Kbytes for data pages.
| | | | With MRU Buffer Replacement Strategy for data pages.
| |
| | TO TABLE
| | #AggTableName
| | Using I/O Size 2 Kbytes for data pages.
更新 2：
一些统计数据：
5M rows 13M rows
CPUTime 263,350 1,180,700
WaitTime 577,574 1,927,399
PhysicalReads 2,304,977 13,704,583
LogicalReads 11,479,123 27,911,085
PagesRead 5,550,737 19,518,030
PhysicalWrites 131,924 5,557,143
PagesWritten 263,640 6,103,708
更新 3：
set statistics io,time on结果重新格式化以便于比较：
5M rows 13M rows
+-------------------------+-----------+------------+
| #AggTableName | | |
| logical reads regular | 81 114 | 248 961 |
| apf | 0 | 0 |
| total | 81 114 | 248 961 |
| physical reads regular | 0 | 2 |
| apf | 0 | 0 |
| total | 0 | 2 |
| apf IOs | 0 | 0 |
+-------------------------+-----------+------------+
| Worktable1 | | |
| logical reads regular | 1 924 136 | 8 200 130 |
| apf | 0 | 0 |
| total | 1 924 136 | 8 200 130 |
| physical reads regular | 1 621 916 | 11 906 846 |
| apf | 0 | 0 |
| total | 1 621 916 | 11 906 846 |
| apf IOs | 0 | 0 |
+-------------------------+-----------+------------+
| TableName | | |
| logical reads regular | 5 651 318 | 13 921 342 |
| apf | 52 | 20 |
| total | 5 651 370 | 13 921 362 |
| physical reads regular | 38 207 | 345 156 |
| apf | 820 646 | 1 768 064 |
| total | 858 853 | 2 113 220 |
| apf IOs | 819 670 | 1 754 171 |
+-------------------------+-----------+------------+
| Total writes | 0 | 5 675 198 |
| Execution time | 4 339 | 18 678 |
| CPU time | 211 321 | 930 657 |
| Elapsed time | 316 396 | 2 719 108 |
+-------------------------+-----------+------------+
5M 行：
Total writes for this command: 0
Execution Time 0.
Adaptive Server cpu time: 0 ms. Adaptive Server elapsed time: 0 ms.
Parse and Compile Time 0.
Adaptive Server cpu time: 0 ms.
Parse and Compile Time 0.
Adaptive Server cpu time: 0 ms.
Table: #AggTableName scan count 0, logical reads: (regular=81114 apf=0 total=81114), physical reads: (regular=0 apf=0 total=0), apf IOs used=0
Table: Worktable1 scan count 1, logical reads: (regular=1924136 apf=0 total=1924136), physical reads: (regular=1621916 apf=0 total=1621916), apf IOs used=0
Table: TableName scan count 1, logical reads: (regular=5651318 apf=52 total=5651370), physical reads: (regular=38207 apf=820646 total=858853), apf IOs used=819670
Total writes for this command: 0
Execution Time 4339.
Adaptive Server cpu time: 211321 ms. Adaptive Server elapsed time: 316396 ms.
13M 行：
Total writes for this command: 0
Execution Time 0.
Adaptive Server cpu time: 0 ms. Adaptive Server elapsed time: 0 ms.
Parse and Compile Time 1.
Adaptive Server cpu time: 50 ms.
Parse and Compile Time 0.
Adaptive Server cpu time: 0 ms.
Table: #AggTableName scan count 0, logical reads: (regular=248961 apf=0 total=248961), physical reads: (regular=2 apf=0 total=2), apf IOs used=0
Table: Worktable1 scan count 1, logical reads: (regular=8200130 apf=0 total=8200130), physical reads: (regular=11906846 apf=0 total=11906846), apf IOs used=0
Table: TableName scan count 1, logical reads: (regular=13921342 apf=20 total=13921362), physical reads: (regular=345156 apf=1768064 total=2113220), apf IOs used=1754171
Total writes for this command: 5675198
Execution Time 18678.
Adaptive Server cpu time: 930657 ms. Adaptive Server elapsed time: 2719108 ms.