2013-05-29：使用最新配置和额外信息更新问题。早些时候我在一个 virtualbox 映像中进行测试。现在我正在生产服务器上进行测试，它更好地反映了现实世界。问题现在应该完全清楚了。如果您以前帮助过我，请仔细阅读
目前我发现 PostgreSQL 中的查询非常慢，但我不明白它怎么会很慢。我把它缩小了一点，所以在这里发布要小得多（而且速度更快，但仍然很慢！）。
小背景：在这个项目中，我有属于用户的广告。用户是该国某个区域的一部分。一个区域可以有多个子区域，因此区域表是一棵树。一个网络被分配给一个区域。在网络上过滤时，它应该过滤该区域及其树中的所有区域子项。因为我无法查询一棵无休止的树，所以我有一张可以压平这棵完整树的表格。
因此，通过 1 个查询(SELECT area_id FROM network_area_flatdeep WHERE network_id = 1)，我得到了属于网络 1 的所有区域：
63, 64, 65, 66, 67, 68, 69, 70
这使得查询非常容易。
慢查询（在测试所有内容之前都已进行 VACUUM ANALYZED）：
EXPLAIN ANALYZE SELECT a0_.id AS id0
FROM advert a0_
INNER JOIN member m6_
ON a0_.user_id = m6_.id
INNER JOIN area a7_
ON m6_.area_id = a7_.id
WHERE a0_.status IN ( 1 )
AND m6_.status IN ( 1 )
AND a7_.id IN (SELECT area_id FROM network_area_flatdeep WHERE network_id IN (1))
ORDER BY a0_.created_date DESC
LIMIT 60;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=41.10..34695.53 rows=60 width=12) (actual time=9.327..134.581 rows=60 loops=1)
-> Nested Loop (cost=41.10..2291276.69 rows=3967 width=12) (actual time=9.326..134.534 rows=60 loops=1)
Join Filter: (a7_.id = m6_.area_id)
Rows Removed by Join Filter: 22566
-> Nested Loop (cost=41.10..821630.16 rows=317633 width=24) (actual time=0.049..39.638 rows=22666 loops=1)
-> Index Scan Backward using advert_created_date_idx on advert a0_ (cost=0.00..762000.64 rows=317633 width=16) (actual time=0.013..4.357 rows=2834 loops=1)
Filter: (status = 1)
Rows Removed by Filter: 21
-> Materialize (cost=41.10..73.38 rows=15 width=8) (actual time=0.000..0.004 rows=8 loops=2834)
-> Nested Loop (cost=41.10..73.30 rows=15 width=8) (actual time=0.031..0.073 rows=8 loops=1)
-> HashAggregate (cost=41.10..41.18 rows=8 width=4) (actual time=0.023..0.026 rows=8 loops=1)
-> Bitmap Heap Scan on network_area_flatdeep (cost=4.37..41.06 rows=15 width=4) (actual time=0.011..0.015 rows=8 loops=1)
Recheck Cond: (network_id = 1)
-> Bitmap Index Scan on idx_c29e880034128b91 (cost=0.00..4.36 rows=15 width=0) (actual time=0.007..0.007 rows=8 loops=1)
Index Cond: (network_id = 1)
-> Index Only Scan using area_pkey on area a7_ (cost=0.00..4.01 rows=1 width=4) (actual time=0.002..0.003 rows=1 loops=8)
Index Cond: (id = network_area_flatdeep.area_id)
Heap Fetches: 8
-> Index Scan using member_pkey on member m6_ (cost=0.00..4.61 rows=1 width=8) (actual time=0.002..0.003 rows=1 loops=22666)
Index Cond: (id = a0_.user_id)
Filter: (status = 1)
Rows Removed by Filter: 0
Total runtime: 134.698 ms
(23 rows)
子查询本身是：
EXPLAIN ANALYZE SELECT area_id FROM network_area_flatdeep WHERE network_id IN (1);
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on network_area_flatdeep (cost=4.37..41.06 rows=15 width=4) (actual time=0.020..0.024 rows=8 loops=1)
Recheck Cond: (network_id = 1)
-> Bitmap Index Scan on idx_c29e880034128b91 (cost=0.00..4.36 rows=15 width=0) (actual time=0.012..0.012 rows=8 loops=1)
Index Cond: (network_id = 1)
Total runtime: 0.051 ms
(5 rows)
这导致：63, 64, 65, 66, 67, 68, 69, 70
因此，我尝试在 id 中进行硬编码。并期望它更快（但不是）：
EXPLAIN ANALYZE SELECT a0_.id AS id0
FROM advert a0_
INNER JOIN member m6_
ON a0_.user_id = m6_.id
INNER JOIN area a7_
ON m6_.area_id = a7_.id
WHERE a0_.status IN ( 1 )
AND m6_.status IN ( 1 )
AND a7_.id IN (63, 64, 65, 66, 67, 68, 69, 70)
ORDER BY a0_.created_date DESC
LIMIT 60;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=17558.82..17558.97 rows=60 width=12) (actual time=56.594..56.670 rows=60 loops=1)
-> Sort (cost=17558.82..17560.07 rows=498 width=12) (actual time=56.593..56.621 rows=60 loops=1)
Sort Key: a0_.created_date
Sort Method: top-N heapsort Memory: 27kB
-> Nested Loop (cost=0.00..17541.62 rows=498 width=12) (actual time=0.047..53.808 rows=4478 loops=1)
-> Nested Loop (cost=0.00..3903.99 rows=286 width=4) (actual time=0.027..17.492 rows=8004 loops=1)
-> Seq Scan on area a7_ (cost=0.00..144.78 rows=8 width=4) (actual time=0.007..0.823 rows=8 loops=1)
Filter: (id = ANY ('{63,64,65,66,67,68,69,70}'::integer[]))
Rows Removed by Filter: 5081
-> Index Scan using idx_70e4fa78bd0f409c on member m6_ (cost=0.00..468.38 rows=152 width=8) (actual time=0.011..1.208 rows=1000 loops=8)
Index Cond: (area_id = a7_.id)
Filter: (status = 1)
Rows Removed by Filter: 2
-> Index Scan using idx_54f1f40ba76ed395 on advert a0_ (cost=0.00..47.49 rows=19 width=16) (actual time=0.002..0.003 rows=1 loops=8004)
Index Cond: (user_id = m6_.id)
Filter: (status = 1)
Rows Removed by Filter: 1
Total runtime: 56.744 ms
(18 rows)
Time: 57.995 ms
我试图将子查询放在 INNER JOIN 中：
EXPLAIN ANALYZE SELECT a0_.id AS id0
FROM advert a0_
INNER JOIN member m6_
ON a0_.user_id = m6_.id
INNER JOIN area a7_
ON m6_.area_id = a7_.id
AND m6_.area_id IN (SELECT area_id FROM network_area_flatdeep WHERE network_id IN (1))
WHERE a0_.status IN ( 1 )
AND m6_.status IN ( 1 )
ORDER BY a0_.created_date DESC
LIMIT 60;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=4.37..34966.73 rows=60 width=12) (actual time=2.957..42.443 rows=60 loops=1)
-> Nested Loop (cost=4.37..2311599.10 rows=3967 width=12) (actual time=2.956..42.394 rows=60 loops=1)
-> Nested Loop Semi Join (cost=4.37..2302173.51 rows=3967 width=20) (actual time=2.949..42.099 rows=60 loops=1)
Join Filter: (m6_.area_id = network_area_flatdeep.area_id)
Rows Removed by Join Filter: 22333
-> Nested Loop (cost=0.00..2230853.09 rows=316797 width=16) (actual time=0.028..18.612 rows=2829 loops=1)
-> Index Scan Backward using advert_created_date_idx on advert a0_ (cost=0.00..762000.64 rows=317633 width=16) (actual time=0.012..3.802 rows=2834 loops=1)
Filter: (status = 1)
Rows Removed by Filter: 21
-> Index Scan using member_pkey on member m6_ (cost=0.00..4.61 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=2834)
Index Cond: (id = a0_.user_id)
Filter: (status = 1)
Rows Removed by Filter: 0
-> Materialize (cost=4.37..41.14 rows=15 width=4) (actual time=0.000..0.004 rows=8 loops=2829)
-> Bitmap Heap Scan on network_area_flatdeep (cost=4.37..41.06 rows=15 width=4) (actual time=0.009..0.015 rows=8 loops=1)
Recheck Cond: (network_id = 1)
-> Bitmap Index Scan on idx_c29e880034128b91 (cost=0.00..4.36 rows=15 width=0) (actual time=0.006..0.006 rows=8 loops=1)
Index Cond: (network_id = 1)
-> Index Only Scan using area_pkey on area a7_ (cost=0.00..2.37 rows=1 width=4) (actual time=0.002..0.003 rows=1 loops=60)
Index Cond: (id = m6_.area_id)
Heap Fetches: 60
Total runtime: 42.538 ms
(22 rows)
我试图摆脱子查询并做出正确的 JOIN 语句network_area_flatdeep（我非常喜欢这个版本）：
EXPLAIN ANALYZE SELECT a0_.id AS id0
FROM advert a0_
INNER JOIN member m6_
ON a0_.user_id = m6_.id
INNER JOIN area a7_
ON m6_.area_id = a7_.id
INNER JOIN network_area_flatdeep n14_
ON a7_.id = n14_.area_id
AND ( n14_.network_id IN ( 1 ) )
WHERE a0_.status IN ( 1 )
AND m6_.status IN ( 1 )
ORDER BY a0_.created_date DESC
LIMIT 60;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=30031.18..30031.33 rows=60 width=12) (actual time=62.968..63.045 rows=60 loops=1)
-> Sort (cost=30031.18..30033.51 rows=934 width=12) (actual time=62.967..62.991 rows=60 loops=1)
Sort Key: a0_.created_date
Sort Method: top-N heapsort Memory: 27kB
-> Nested Loop (cost=0.00..29998.92 rows=934 width=12) (actual time=0.157..60.280 rows=4478 loops=1)
-> Nested Loop (cost=0.00..4401.66 rows=536 width=4) (actual time=0.029..20.488 rows=8004 loops=1)
-> Nested Loop (cost=0.00..120.69 rows=15 width=8) (actual time=0.015..0.084 rows=8 loops=1)
-> Index Scan using idx_c29e880034128b91 on network_area_flatdeep n14_ (cost=0.00..60.47 rows=15 width=4) (actual time=0.009..0.019 rows=8 loops=1)
Index Cond: (network_id = 1)
-> Index Only Scan using area_pkey on area a7_ (cost=0.00..4.01 rows=1 width=4) (actual time=0.004..0.005 rows=1 loops=8)
Index Cond: (id = n14_.area_id)
Heap Fetches: 8
-> Index Scan using idx_70e4fa78bd0f409c on member m6_ (cost=0.00..283.88 rows=152 width=8) (actual time=0.011..1.278 rows=1000 loops=8)
Index Cond: (area_id = a7_.id)
Filter: (status = 1)
Rows Removed by Filter: 2
-> Index Scan using idx_54f1f40ba76ed395 on advert a0_ (cost=0.00..47.57 rows=19 width=16) (actual time=0.003..0.003 rows=1 loops=8004)
Index Cond: (user_id = m6_.id)
Filter: (status = 1)
Rows Removed by Filter: 1
Total runtime: 63.125 ms
(21 rows)
更改IN为=也无济于事。
我尝试按照 wilderplasser 的建议使用 EXISTS，并在下面给出答案。
当我删除 ORDER BY 或网络条件时，查询速度很快（2 毫秒）。但是为什么他们不能一起玩得很好呢？
现在……当我将 network_id 从“1”更改为“10”时。查询速度非常快，就像我想要的那样。10 是包含所有区域的根网络。因此，显然 JOIN 需要过滤掉的结果越少，查询就越快。
结构如下所示：
area (mapped to network 10 which contains 5089 areas in flat table and is FAST)
| |---- area 1 (network 1 which contains 8 areas in flat table and is SLOW)
| |--- more areas
|-- ALOT MORE areas
|-- etc
将网络 1 更改为 10 给出：3.265 ms
总结一下区别：
网络 1 有 8 个区域network_area_flatdeep
网络 10 有 5089 个区域network_area_flatdeep
所以使用网络 1 的 INNER JOIN 非常慢，使用网络 10 的 INNER JOIN 非常快。子查询的行为相同。
广告表
353804 rows
Table "public.advert"
Column | Type | Modifiers | Storage | Stats target | Description
-----------------------------+--------------------------------+-----------------------------------------------------+----------+--------------+-------------
id | integer | not null default nextval('advert_id_seq'::regclass) | plain | |
user_id | integer | not null | plain | |
advert_category_id | integer | not null | plain | |
currency_id | integer | not null | plain | |
advert_kind_id | integer | not null | plain | |
advert_price_id | integer | | plain | |
external_source_id | integer | | plain | |
status | integer | not null | plain | |
type | integer | not null | plain | |
title | character varying(60) | not null | extended | |
description | text | not null | extended | |
price | numeric(19,2) | default NULL::numeric | main | |
accepting_bids | boolean | not null | plain | |
promoted | boolean | not null | plain | |
edited_date | timestamp(0) without time zone | default NULL::timestamp without time zone | plain | |
created_date | timestamp(0) without time zone | not null | plain | |
archived_date | timestamp(0) without time zone | default NULL::timestamp without time zone | plain | |
views | integer | not null | plain | |
checked_date | timestamp(0) without time zone | default NULL::timestamp without time zone | plain | |
archived_by_cron | boolean | not null | plain | |
unarchived_by_cron | boolean | not null | plain | |
containting_forbidden_words | boolean | not null | plain | |
external_id | character varying(255) | default NULL::character varying | extended | |
new_product | boolean | not null | plain | |
Indexes:
"advert_pkey" PRIMARY KEY, btree (id)
"advert_external_idx_uq" UNIQUE, btree (external_id, external_source_id)
"advert_archived_date_idx" btree (archived_date)
"advert_checked_date_idx" btree (checked_date)
"advert_created_date_idx" btree (created_date)
"advert_edited_date_idx" btree (edited_date)
"advert_external_id_idx" btree (external_id)
"advert_price_idx" btree (price)
"advert_status_idx" btree (status)
"advert_type_idx" btree (type)
"advert_views_idx" btree (views)
"idx_54f1f40b38248176" btree (currency_id)
"idx_54f1f40b54b67d66" btree (advert_price_id)
"idx_54f1f40b9a2e6cff" btree (advert_kind_id)
"idx_54f1f40ba76ed395" btree (user_id)
"idx_54f1f40bb167b375" btree (external_source_id)
"idx_54f1f40bd4436821" btree (advert_category_id)
Foreign-key constraints:
"fk_54f1f40b38248176" FOREIGN KEY (currency_id) REFERENCES currency(id) ON DELETE RESTRICT
"fk_54f1f40b54b67d66" FOREIGN KEY (advert_price_id) REFERENCES advertprice(id) ON DELETE RESTRICT
"fk_54f1f40b9a2e6cff" FOREIGN KEY (advert_kind_id) REFERENCES advertkind(id) ON DELETE RESTRICT
"fk_54f1f40ba76ed395" FOREIGN KEY (user_id) REFERENCES member(id) ON DELETE CASCADE
"fk_54f1f40bb167b375" FOREIGN KEY (external_source_id) REFERENCES externalsource(id) ON DELETE RESTRICT
"fk_54f1f40bd4436821" FOREIGN KEY (advert_category_id) REFERENCES advertcategory(id) ON DELETE RESTRICT
Referenced by:
TABLE "advert_photo" CONSTRAINT "fk_1c939974d07eccb6" FOREIGN KEY (advert_id) REFERENCES advert(id) ON DELETE CASCADE
TABLE "banner" CONSTRAINT "fk_6f9db8e7d07eccb6" FOREIGN KEY (advert_id) REFERENCES advert(id) ON DELETE SET NULL
TABLE "advertbid" CONSTRAINT "fk_fccdba75d07eccb6" FOREIGN KEY (advert_id) REFERENCES advert(id) ON DELETE CASCADE
Has OIDs: no
面积表：
5089 rows
Table "public.area"
Column | Type | Modifiers | Storage | Stats target | Description
------------+---------+---------------------------------------------------+---------+--------------+-------------
id | integer | not null default nextval('area_id_seq'::regclass) | plain | |
network_id | integer | | plain | |
parent_id | integer | | plain | |
selectable | boolean | not null | plain | |
Indexes:
"area_pkey" PRIMARY KEY, btree (id)
"idx_d7943d6834128b91" btree (network_id)
"idx_d7943d68727aca70" btree (parent_id)
Foreign-key constraints:
"fk_d7943d6834128b91" FOREIGN KEY (network_id) REFERENCES network(id) ON DELETE RESTRICT
"fk_d7943d68727aca70" FOREIGN KEY (parent_id) REFERENCES area(id) ON DELETE CASCADE
Referenced by:
TABLE "network_area_flat" CONSTRAINT "fk_10aae5b2bd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE CASCADE
TABLE "area_language" CONSTRAINT "fk_17d42f7dbd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE CASCADE
TABLE "area_zip_code" CONSTRAINT "fk_62a3bf90bd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE CASCADE
TABLE "member" CONSTRAINT "fk_70e4fa78bd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE RESTRICT
TABLE "network_area_flatdeep" CONSTRAINT "fk_c29e8800bd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE CASCADE
TABLE "area" CONSTRAINT "fk_d7943d68727aca70" FOREIGN KEY (parent_id) REFERENCES area(id) ON DELETE CASCADE
Has OIDs: no
成员表：
182450 rows
Table "public.member"
Column | Type | Modifiers | Storage | Stats target | Description
-----------------------+--------------------------------+-----------------------------------------------------+----------+--------------+-------------
id | integer | not null default nextval('member_id_seq'::regclass) | plain | |
language_id | integer | not null | plain | |
area_id | integer | not null | plain | |
company_id | integer | | plain | |
external_source_id | integer | | plain | |
email | character varying(255) | not null | extended | |
password | character varying(40) | not null | extended | |
status | integer | not null | plain | |
name | character varying(150) | not null | extended | |
zip_code | character varying(20) | | extended | |
phone_number | character varying(120) | default NULL::character varying | extended | |
using_email_service | boolean | not null | plain | |
edited_date | timestamp(0) without time zone | default NULL::timestamp without time zone | plain | |
created_date | timestamp(0) without time zone | not null | plain | |
hiding_on_own_network | boolean | not null | plain | |
staff | boolean | not null | plain | |
superuser | boolean | not null | plain | |
external_id | character varying(255) | default NULL::character varying | extended | |
last_login_date | timestamp(0) without time zone | default NULL::timestamp without time zone | plain | |
deleted_adverts | integer | not null | plain | |
Indexes:
"member_pkey" PRIMARY KEY, btree (id)
"user_email_idx_uq" UNIQUE, btree (email)
"user_external_idx_uq" UNIQUE, btree (external_id, external_source_id)
"idx_70e4fa7882f1baf4" btree (language_id)
"idx_70e4fa78979b1ad6" btree (company_id)
"idx_70e4fa78b167b375" btree (external_source_id)
"idx_70e4fa78bd0f409c" btree (area_id)
"user_external_id_idx" btree (external_id)
"user_name_idx" btree (name)
"user_status_idx" btree (status)
Foreign-key constraints:
"fk_70e4fa7882f1baf4" FOREIGN KEY (language_id) REFERENCES language(id) ON DELETE RESTRICT
"fk_70e4fa78979b1ad6" FOREIGN KEY (company_id) REFERENCES company(id) ON DELETE SET NULL
"fk_70e4fa78b167b375" FOREIGN KEY (external_source_id) REFERENCES externalsource(id) ON DELETE RESTRICT
"fk_70e4fa78bd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE RESTRICT
Referenced by:
TABLE "user_link" CONSTRAINT "fk_4c2dd538a76ed395" FOREIGN KEY (user_id) REFERENCES member(id) ON DELETE CASCADE
TABLE "advert" CONSTRAINT "fk_54f1f40ba76ed395" FOREIGN KEY (user_id) REFERENCES member(id) ON DELETE CASCADE
TABLE "banner" CONSTRAINT "fk_6f9db8e7a76ed395" FOREIGN KEY (user_id) REFERENCES member(id) ON DELETE SET NULL
TABLE "user_admin_module_permission" CONSTRAINT "fk_74fee7cea76ed395" FOREIGN KEY (user_id) REFERENCES member(id) ON DELETE CASCADE
TABLE "user_admin_resource_permission" CONSTRAINT "fk_c9fcf279a76ed395" FOREIGN KEY (user_id) REFERENCES member(id) ON DELETE CASCADE
Has OIDs: no
Network_area_flatdeep 表：
10177 rows
Table "public.network_area_flatdeep"
Column | Type | Modifiers | Storage | Stats target | Description
--------------+--------------------------------+--------------------------------------------------------------------+---------+--------------+-------------
id | integer | not null default nextval('network_area_flatdeep_id_seq'::regclass) | plain | |
network_id | integer | not null | plain | |
area_id | integer | not null | plain | |
created_date | timestamp(0) without time zone | not null | plain | |
Indexes:
"network_area_flatdeep_pkey" PRIMARY KEY, btree (id)
"area_flatdeep_idx_uq" UNIQUE, btree (area_id, network_id, created_date)
"idx_c29e880034128b91" btree (network_id)
"idx_c29e8800bd0f409c" btree (area_id)
Foreign-key constraints:
"fk_c29e880034128b91" FOREIGN KEY (network_id) REFERENCES network(id) ON DELETE CASCADE
"fk_c29e8800bd0f409c" FOREIGN KEY (area_id) REFERENCES area(id) ON DELETE CASCADE
Has OIDs: no
简短的服务器配置：
version
--------------------------------------------------------------------------------------------------------------
PostgreSQL 9.2.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-3), 64-bit
name | current_setting | source
----------------------------+--------------------+----------------------
shared_buffers | 1800MB | configuration file
work_mem | 4MB | configuration file
结论：
数据越多（过滤掉的越少），查询速度就越快。我现在有点不知所措。PostgreSQL 怎么会这么慢？再说一遍：40ms 看起来并不是特别慢，但是对于带有大 SELECT 语句的实际查询，查询大部分时间是 1.5 秒，最多可能需要 3 秒。
所有过滤都是在索引上完成的。排序也是在索引上完成的。
有谁知道如何改进这个我显然需要分离数据的简单过滤器？