1

我遇到了大型查询的问题，我希望依赖我的 postgresql.config 的错误配置。我的设置是 Ubuntu 17.10 上的 PostgreSQL 9.6，具有 32GB RAM 和 3TB HDD。该查询正在运行 pgr_dijkstraCost以在 25.000 个链接的网络中创建约 10.000 个点的 OD 矩阵。因此，结果表预计会非常大（~100'000'000 行，列从、到、成本）。但是，创建简单测试select x,1 as c2,2 as c3 from generate_series(1,90000000)成功。

查询计划：

                                     QUERY PLAN                                      
--------------------------------------------------------------------------------------
 Function Scan on pgr_dijkstracost  (cost=393.90..403.90 rows=1000 width=24)
   InitPlan 1 (returns $0)
     ->  Aggregate  (cost=196.82..196.83 rows=1 width=32)
           ->  Seq Scan on building_nodes b  (cost=0.00..166.85 rows=11985 width=4)
   InitPlan 2 (returns $1)
     ->  Aggregate  (cost=196.82..196.83 rows=1 width=32)
           ->  Seq Scan on building_nodes b_1  (cost=0.00..166.85 rows=11985 width=4)

这会导致 PostgreSQL 崩溃：

WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the 
current transaction and exit, because another server process exited      
normally and possibly corrupted shared memory.

运行dmesg我可以追溯到内存不足问题：

Out of memory: Kill process 5630 (postgres) score 949 or sacrifice child
[ 5322.821084] Killed process 5630 (postgres) total-vm:36365660kB,anon-rss:32344260kB, file-rss:0kB, shmem-rss:0kB
[ 5323.615761] oom_reaper: reaped process 5630 (postgres), now anon-rss:0kB,file-rss:0kB, shmem-rss:0kB
[11741.155949] postgres invoked oom-killer: gfp_mask=0x14201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), nodemask=(null),  order=0, oom_score_adj=0
[11741.155953] postgres cpuset=/ mems_allowed=0

运行查询时，我还可以观察top到我的 RAM 在崩溃前下降到 0。崩溃前提交的内存量：

$grep Commit /proc/meminfo
CommitLimit:    18574304 kB
Committed_AS:   42114856 kB

当 RAM 不足时，我希望 HDD 用于写入/缓冲临时数据。但是我的硬盘上的可用空间在处理过程中不会改变。所以我开始挖掘丢失的配置（由于我重新定位的数据目录而导致的问题）并关注不同的站点：

https://www.postgresql.org/docs/current/static/kernel-resources.html#LINUX-MEMORY-OVERCOMMIT https://www.credativ.com/credativ-blog/2010/03/postgresql-and-linux -内存管理

除了数据目录中的更改外，我对 postgresql.conf 的原始设置是默认设置：

data_directory = '/hdd_data/postgresql/9.6/main'
shared_buffers = 128MB          # min 128kB
#huge_pages = try           # on, off, or try
#temp_buffers = 8MB         # min 800kB
#max_prepared_transactions = 0      # zero disables the feature
#work_mem = 4MB             # min 64kB
#maintenance_work_mem = 64MB        # min 1MB
#replacement_sort_tuples = 150000   # limits use of replacement selection sort
#autovacuum_work_mem = -1       # min 1MB, or -1 to use maintenance_work_mem
#max_stack_depth = 2MB          # min 100kB
dynamic_shared_memory_type = posix  # the default is the first option

我改变了配置：

shared_buffers = 128MB
work_mem = 40MB             # min 64kB
maintenance_work_mem = 64MB

重新启动sudo service postgresql reload并测试相同的查询，但发现行为没有变化。这是否仅仅意味着，这么大的查询不能做？任何帮助表示赞赏。

4

0 回答 0