我们有一个复杂的查询,它根据客户的不同选项动态构建并运行数据查询。我们在 Azure 中有一些函数,它们每晚运行这些查询来构建报告数据,我们运行大约。其中30k。孤立的查询大约是我能得到的最快的,大约 100 毫秒,但是当我们在 Azure 中的消费计划上并行运行函数时(限制为同时运行最多 5 个函数),查询的性能是下降,有些甚至在 5 分钟后超时,有些在我单独测试过的超时,并且在 100 毫秒以下进入。没有写入,因为这是使用 Azure 中的只读副本加载此数据。
我们在虚拟机上使用 PgBouncer 在托管 Azure 上运行 Postgres 11.6。所有这些查询都将转到配置为 4 vCore Memory Optimized 的只读副本。
我们可以进行哪些更改以允许更多并行执行这些查询,或者扩大我们唯一的选择?
我想分享解释分析,但这受业务限制。请让我知道哪些信息会有所帮助,我会尽量提供。
CTE Scan on bravo_zulu romeo (cost=2151.89..2151.94 rows=1 width=204) (actual time=27.756..84.147 rows=36 loops=1)
CTE bravo_zulu
-> Nested Loop (cost=13.84..2151.89 rows=1 width=139) (actual time=27.744..84.009 rows=36 loops=1)
-> Nested Loop (cost=13.42..2151.43 rows=1 width=139) (actual time=26.811..76.983 rows=36 loops=1)
-> Nested Loop (cost=12.86..130.51 rows=1 width=44) (actual time=7.471..19.361 rows=29 loops=1)
-> Nested Loop (cost=4.88..97.73 rows=1 width=24) (actual time=7.410..10.480 rows=24 loops=1)
-> Index Scan using yankee on xray_zulu foxtrot_uniform (cost=0.28..8.29 rows=1 width=8) (actual time=1.339..1.340 rows=1 loops=1)
Index Cond: ("juliet" = 20)
-> Bitmap Heap Scan on golf_delta hotel_six (cost=4.60..89.43 rows=1 width=20) (actual time=6.064..9.123 rows=24 loops=1)
Recheck Cond: ("delta_oscar_hotel" = foxtrot_uniform."lima")
Filter: ("juliet" = ANY ('foxtrot_oscar'::integer[]))
Rows Removed by Filter: 442
Heap Blocks: exact=65
-> Bitmap Index Scan on papa (cost=0.00..4.60 rows=42 width=0) (actual time=0.024..0.024 rows=466 loops=1)
Index Cond: ("delta_oscar_hotel" = foxtrot_uniform."lima")
-> Bitmap Heap Scan on delta_sierra_two bravo_hotel (cost=7.98..32.76 rows=2 width=20) (actual time=0.321..0.363 rows=1 loops=24)
Recheck Cond: ((hotel_six."juliet" = "xray_india") OR (hotel_six."juliet" = "foxtrot_foxtrot"))
Filter: ("hotel_golf" = 23)
Rows Removed by Filter: 10
Heap Blocks: exact=240
-> BitmapOr (cost=7.98..7.98 rows=9 width=0) (actual time=0.066..0.066 rows=0 loops=24)
-> Bitmap Index Scan on delta_sierra_sierra (cost=0.00..3.99 rows=5 width=0) (actual time=0.063..0.063 rows=11 loops=24)
Index Cond: (hotel_six."juliet" = "xray_india")
-> Bitmap Index Scan on xray_sierra (cost=0.00..3.99 rows=4 width=0) (actual time=0.002..0.002 rows=0 loops=24)
Index Cond: (hotel_six."juliet" = "foxtrot_foxtrot")
-> Index Only Scan using echo on xray_papa victor (cost=0.56..2020.44 rows=48 width=102) (actual time=1.606..1.986 rows=1 loops=29)
Index Cond: (("five_lima" = 23) AND ("seven_yankee" = bravo_hotel."november") AND ("charlie_hotel" five_romeo NULL))
Filter: (("three" = 'charlie_romeo'::text) AND (("alpha" = 'golf_bravo'::text) OR ("alpha" = 'delta_echo'::text)) AND ((("alpha" = ANY ('mike_juliet'::text[])) AND ("mike_lima" >= 'xray_whiskey'::date) AND ("mike_lima" <= 'uniform'::date)) OR (("alpha" = ANY ('kilo'::text[])) AND ("quebec_uniform" >= 'xray_whiskey'::date) AND ("quebec_uniform" <= 'uniform'::date)) OR (("alpha" = 'quebec_alpha_quebec'::text) AND ("quebec_uniform" >= 'xray_whiskey'::date) AND ("quebec_uniform" <= 'uniform'::date) AND ("mike_lima" >= 'xray_whiskey'::date) AND ("mike_lima" <= 'uniform'::date))) AND ((("alpha" = ANY ('oscar'::text[])) AND ("seven_india" = ANY ('four'::text[]))) OR (("alpha" = ANY ('quebec_alpha_delta'::text[])) AND ("seven_charlie" = ANY ('four'::text[])))))
Rows Removed by Filter: 1059
Heap Fetches: 0
-> Index Scan using bravo_papa on tango sierra (cost=0.42..0.45 rows=1 width=16) (actual time=0.194..0.194 rows=1 loops=36)
Index Cond: (("bravo_two" = 23) AND ("delta_tango" = six1."delta_oscar_romeo"))
SubPlan
-> Result (cost=0.01..0.02 rows=1 width=32) (actual time=0.001..0.001 rows=1 loops=36)
One-Time Filter: ((romeo.zulu = 'golf_bravo'::text) AND (romeo.golf_uniform = 20) AND (romeo.charlie_two = 'charlie_romeo'::text))
SubPlan
-> Result (cost=0.01..0.02 rows=1 width=32) (actual time=0.000..0.000 rows=0 loops=36)
One-Time Filter: ((romeo.zulu = 'delta_echo'::text) AND (romeo.charlie_two = 'charlie_romeo'::text) AND (romeo.golf_uniform = 20))
Planning time: 19.385 ms
Execution time: 84.373 ms
上面是一个匿名执行计划,在 Azure 中并行运行函数时,这个相同的查询超时。
表大小并不大,最大的是 8m 行,但所有其他的都低 100k。