HOT 和 FILLFACTOR 结果
我有一些高 UPDATE 表,我已将其调整FILLFACTOR
为 95%,我正在重新检查它们。我认为我的设置不正确,并且不清楚如何智能地调整它们。我又通过了 Laurenz Albe 关于 HOT 更新的有用博客文章
https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/
...以及清晰的源代码阅读我:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/heap/README.HOT
下面是一个查询,改编自博客文章,用于检查系统中表的状态,以及一些示例输出:
SELECT relname,
n_tup_upd as total_update_count,
n_tup_hot_upd as hot_update_count,
coalesce(div_safe(n_tup_upd, n_tup_hot_upd),0) as total_by_hot,
coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0) as hot_by_total
FROM pg_stat_user_tables
order by 4 desc;
几个结果:
relname total_update_count hot_update_count total_by_hot hot_by_total
rollups 369418 128 2886.0781 0.00034649097
q_event 71781 541 132.68207 0.007536813
analytic_scan 2104727 34304 61.35515 0.016298551
clinic 4424 77 57.454544 0.017405063
facility_location 179636 6489 27.683157 0.03612305
target_snapshot 494 18 27.444445 0.036437247
inv 1733021 78234 22.151762 0.045143135
我不确定我在这里寻找什么比率。谁能告诉我如何阅读这些结果,或者阅读什么来弄清楚如何解释它们?
这些更新是 HOTable 吗?
我没有在这个问题的原始草案中解决这个基本点。几个月前我检查了我的补丁,然后我跑了SET (fillfactor = 95)
,然后VACUUM (FULL, VERBOSE, ANALYZE)
在我的 13 张桌子上跑。(VERBOSE
因为我有一些桌子VACUUM
因为需要清理几个月的流程而无法使用,这就是我发现问题的方式。pg_stat_activity
是我的朋友。)
但是,至少大多数都触及索引列......但具有相同的值。像1 = 1
,所以值没有变化。我一直认为那是 HOTable。如果我错了,那就糟透了。如果不是,我主要希望澄清 , 和 之间关系的确切目标fillfactor
是n_tup_upd
什么n_tup_hot_upd
。
SELECT relname,
n_tup_upd as total_update_count,
n_tup_hot_upd as hot_update_count,
coalesce(div_safe(n_tup_upd, n_tup_hot_upd),0) as total_by_hot,
coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0) as hot_by_total,
(select value::integer from table_get_options('data',relname) where option = 'fillfactor') as fillfactor_setting
FROM pg_stat_user_tables
WHERE relname IN (
'activity',
'analytic_productivity',
'analytic_scan',
'analytic_sterilizer_load',
'analytic_sterilizer_loadinv',
'analytic_work',
'assembly',
'data_file_info',
'inv',
'item',
'print_job',
'q_event')
order by 4 desc;
结果:
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| relname | total_update_count | hot_update_count | total_divided_by_hot | hot_divided_by_total | fillfactor_setting |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| q_event | 71810 | 553 | 129.85533 | 0.0077008773 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_scan | 2109206 | 34536 | 61.072678 | 0.016373934 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| inv | 1733176 | 78387 | 22.110502 | 0.045227375 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| item | 630586 | 32110 | 19.638306 | 0.05092089 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_loadinv | 76976539 | 5206806 | 14.783831 | 0.06764147 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_work | 8117050 | 608847 | 13.331839 | 0.07500841 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| assembly | 90580 | 7281 | 12.4405985 | 0.08038198 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_load | 19249 | 2997 | 6.422756 | 0.1556964 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| activity | 3795 | 711 | 5.3375525 | 0.18735178 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_productivity | 106486 | 25899 | 4.1115875 | 0.24321507 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| print_job | 1414 | 388 | 3.6443298 | 0.27439886 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| data_file_info | 402086 | 285663 | 1.4075537 | 0.7104525 | 90 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
(我只是在https://www.tablesgenerator.com/text_tables寻找并找到了一个在线表格生成器来帮助解决这种示例。使用起来有点尴尬,但比手动构建等宽对齐文本更快.)
FILLFACTOR 和 HOT 更新率
我想我可以通过改编来自https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/的 Laurenz Albe 的代码来解决这个问题。我在这里所做的只是制作一个脚本,用FILLFACTOR
10、20、30 .....100% 构建一个表,然后以相同的方式为每个百分比更新它。每次创建表时,都会填充 256 条记录,然后每条记录更新 10 次。更新将一个非索引字段设置回自身,因此实际上没有值更改:
UPDATE mytable SET val = val;
以下是结果:
+------------+---------------+-------------+--------------+
| FILLFACTOR | total_updates | hot_updates | total_to_hot |
+------------+---------------+-------------+--------------+
| 10 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 20 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 30 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 40 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 50 | 2350 | 2223 | 1.06 |
+------------+---------------+-------------+--------------+
| 60 | 2350 | 2188 | 1.07 |
+------------+---------------+-------------+--------------+
| 70 | 2350 | 1883 | 1.25 |
+------------+---------------+-------------+--------------+
| 80 | 2350 | 1574 | 1.49 |
+------------+---------------+-------------+--------------+
| 90 | 2350 | 1336 | 1.76 |
+------------+---------------+-------------+--------------+
| 100 | 2350 | 987 | 2.38 |
+------------+---------------+-------------+--------------+
由此看来,当total_to_hot
比率上升时,增加FILLFACTOR
.
https://www.postgresql.org/docs/13/monitoring-stats.html
n_tup_upd
计算所有更新,包括 HOT 更新,并且n_tup_hot_upd
只计算 HOT 更新。但这似乎不算“如果我们没有用完页面上的空间,这可能是一个热门更新”。那会很棒,但似乎也有很多要求。(也许跟踪它的成本更高是合理的?)
这是脚本。我对每个FILLFACTOR
.
-- Set up the table for the test
DROP TABLE IF EXISTS mytable;
CREATE TABLE mytable (
id integer PRIMARY KEY,
val integer NOT NULL
) WITH (autovacuum_enabled = off);
-- Change the FILLFACTOR. The default is 100.
ALTER TABLE mytable SET (fillfactor = 10); -- The only part that changes between runs.
-- Seed the data
INSERT INTO mytable
SELECT *, 0
FROM generate_series(1, 235) AS n;
-- Thrash the data
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
-- How does it look?
SELECT n_tup_upd as total_updates,
n_tup_hot_upd as hot_updates,
div_safe(n_tup_upd, n_tup_hot_upd) as total_to_hot
FROM pg_stat_user_tables
WHERE relname = 'mytable';
检查 FILLFACTOR 设置
作为旁注,我想快速打电话检查FILLFACTOR
桌子上的设置,结果发现它比我想象的要复杂。我写了一个有效的函数,但可能会看到一些改进......如果有人有建议的话。我这样称呼它:
select * from table_get_options('foo', 'bar');
或者
select * from table_get_options('foo','bar') where option = 'fillfactor';
这是代码,如果有人可以提供改进:
CREATE OR REPLACE FUNCTION dba.table_get_options(text,text)
RETURNS TABLE (
schema_name text,
table_name text,
option text,
value text
)
LANGUAGE SQL AS
$BODY$
WITH
packed_options AS (
select pg_class.relname as table_name,
btrim(pg_options_to_table(pg_class.reloptions)::text, '()') as option_kvp -- Convert to text (fillfactor,95), then strip off ( and )
from pg_class
join pg_namespace
on pg_namespace.oid = pg_class.relnamespace
where pg_namespace.nspname = $1
and relname = $2
and reloptions is not null
),
unpacked_options AS (
select $1 as schema_name,
$2 as table_name,
split_part(option_kvp, ',', 1) as option,
split_part(option_kvp, ',', 2) as value
from packed_options
)
select * from unpacked_options;
$BODY$;