0

HOT 和 FILLFACTOR 结果

我有一些高 UPDATE 表,我已将其调整FILLFACTOR为 95%,我正在重新检查它们。我认为我的设置不正确,并且不清楚如何智能地调整它们。我又通过了 Laurenz Albe 关于 HOT 更新的有用博客文章

https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/

...以及清晰的源代码阅读我:

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/heap/README.HOT

下面是一个查询,改编自博客文章,用于检查系统中表的状态,以及一些示例输出:

SELECT relname,
       n_tup_upd                                          as total_update_count,
       n_tup_hot_upd                                      as hot_update_count,
       coalesce(div_safe(n_tup_upd,     n_tup_hot_upd),0) as total_by_hot,
       coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0)     as hot_by_total
                                      
FROM pg_stat_user_tables
order by 4 desc;

几个结果:

relname            total_update_count    hot_update_count    total_by_hot   hot_by_total
rollups                        369418                 128    2886.0781      0.00034649097
q_event                         71781                 541    132.68207      0.007536813
analytic_scan                 2104727               34304     61.35515      0.016298551
clinic                           4424                  77     57.454544     0.017405063
facility_location              179636                6489     27.683157     0.03612305
target_snapshot                   494                  18     27.444445     0.036437247
inv                           1733021               78234     22.151762     0.045143135

我不确定我在这里寻找什么比率。谁能告诉我如何阅读这些结果,或者阅读什么来弄清楚如何解释它们?

这些更新是 HOTable 吗?

我没有在这个问题的原始草案中解决这个基本点。几个月前我检查了我的补丁,然后我跑了SET (fillfactor = 95),然后VACUUM (FULL, VERBOSE, ANALYZE)在我的 13 张桌子上跑。(VERBOSE因为我有一些桌子VACUUM因为需要清理几个月的流程而无法使用,这就是我发现问题的方式。pg_stat_activity是我的朋友。)

但是,至少大多数都触及索引列......但具有相同的值。像1 = 1,所以值没有变化。我一直认为那是 HOTable。如果我错了,那就糟透了。如果不是,我主要希望澄清 , 和 之间关系的确切目标fillfactorn_tup_upd什么n_tup_hot_upd

SELECT relname,
       n_tup_upd                                          as total_update_count,
       n_tup_hot_upd                                      as hot_update_count,
       coalesce(div_safe(n_tup_upd,     n_tup_hot_upd),0) as total_by_hot,
       coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0)     as hot_by_total,
       (select value::integer from table_get_options('data',relname) where option = 'fillfactor') as fillfactor_setting
                         
FROM pg_stat_user_tables
WHERE relname IN  (
    'activity',
    'analytic_productivity',
    'analytic_scan',
    'analytic_sterilizer_load',
    'analytic_sterilizer_loadinv',
    'analytic_work',
    'assembly',
    'data_file_info',
    'inv',
    'item',
    'print_job',
    'q_event')
    
order by 4 desc;

结果:

+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| relname                     | total_update_count | hot_update_count | total_divided_by_hot | hot_divided_by_total | fillfactor_setting |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| q_event                     |              71810 |              553 |            129.85533 |         0.0077008773 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_scan               |            2109206 |            34536 |            61.072678 |          0.016373934 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| inv                         |            1733176 |            78387 |            22.110502 |          0.045227375 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| item                        |             630586 |            32110 |            19.638306 |           0.05092089 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_loadinv |           76976539 |          5206806 |            14.783831 |           0.06764147 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_work               |            8117050 |           608847 |            13.331839 |           0.07500841 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| assembly                    |              90580 |             7281 |           12.4405985 |           0.08038198 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_load    |              19249 |             2997 |             6.422756 |            0.1556964 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| activity                    |               3795 |              711 |            5.3375525 |           0.18735178 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_productivity       |             106486 |            25899 |            4.1115875 |           0.24321507 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| print_job                   |               1414 |              388 |            3.6443298 |           0.27439886 |                 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| data_file_info              |             402086 |           285663 |            1.4075537 |            0.7104525 |                 90 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+

(我只是在https://www.tablesgenerator.com/text_tables寻找并找到了一个在线表格生成器来帮助解决这种示例。使用起来有点尴尬,但比手动构建等宽对齐文本更快.)

FILLFACTOR 和 HOT 更新率

我想我可以通过改编来自https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/的 Laurenz Albe 的代码来解决这个问题。我在这里所做的只是制作一个脚本,用FILLFACTOR10、20、30 .....100% 构建一个表,然后以相同的方式为每个百分比更新它。每次创建表时,都会填充 256 条记录,然后每条记录更新 10 次。更新将一个非索引字段设置回自身,因此实际上没有值更改:

UPDATE mytable SET val = val;

以下是结果:

+------------+---------------+-------------+--------------+
| FILLFACTOR | total_updates | hot_updates | total_to_hot |
+------------+---------------+-------------+--------------+
|         10 |          2350 |        2350 |         1.00 |
+------------+---------------+-------------+--------------+
|         20 |          2350 |        2350 |         1.00 |
+------------+---------------+-------------+--------------+
|         30 |          2350 |        2350 |         1.00 |
+------------+---------------+-------------+--------------+
|         40 |          2350 |        2350 |         1.00 |
+------------+---------------+-------------+--------------+
|         50 |          2350 |        2223 |         1.06 |
+------------+---------------+-------------+--------------+
|         60 |          2350 |        2188 |         1.07 |
+------------+---------------+-------------+--------------+
|         70 |          2350 |        1883 |         1.25 |
+------------+---------------+-------------+--------------+
|         80 |          2350 |        1574 |         1.49 |
+------------+---------------+-------------+--------------+
|         90 |          2350 |        1336 |         1.76 |
+------------+---------------+-------------+--------------+
|        100 |          2350 |         987 |         2.38 |
+------------+---------------+-------------+--------------+

由此看来,当total_to_hot比率上升,增加FILLFACTOR.

https://www.postgresql.org/docs/13/monitoring-stats.html

n_tup_upd计算所有更新,包括 HOT 更新,并且n_tup_hot_upd只计算 HOT 更新。但这似乎不算“如果我们没有用完页面上的空间,这可能是一个热门更新”。那会很棒,但似乎也有很多要求。(也许跟踪它的成本更高是合理的?)

这是脚本。我对每个FILLFACTOR.

-- Set up the table for the test
DROP TABLE IF EXISTS mytable;

CREATE TABLE mytable (
   id  integer PRIMARY KEY,
   val integer NOT NULL
) WITH (autovacuum_enabled = off);
 
-- Change the FILLFACTOR. The default is 100.
ALTER TABLE mytable SET (fillfactor = 10);  -- The only part that changes between runs.

-- Seed the data
INSERT INTO mytable
SELECT *, 0
FROM generate_series(1, 235) AS n;

-- Thrash the data
UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

UPDATE mytable SET val = val;
SELECT pg_sleep(1);

-- How does it look?
SELECT n_tup_upd                          as total_updates,
       n_tup_hot_upd                      as hot_updates, 
       div_safe(n_tup_upd, n_tup_hot_upd) as total_to_hot
      
  FROM pg_stat_user_tables 
  
 WHERE relname = 'mytable';

检查 FILLFACTOR 设置

作为旁注,我想快速打电话检查FILLFACTOR桌子上的设置,结果发现它比我想象的要复杂。我写了一个有效的函数,但可能会看到一些改进......如果有人有建议的话。我这样称呼它:

select * from table_get_options('foo', 'bar');

或者

select * from table_get_options('foo','bar') where option = 'fillfactor';

这是代码,如果有人可以提供改进:

CREATE OR REPLACE FUNCTION dba.table_get_options(text,text)
  RETURNS TABLE (
    schema_name text,
    table_name  text,
    option      text,
    value       text
)

LANGUAGE SQL AS

$BODY$
WITH
packed_options AS (
select pg_class.relname as table_name,
       btrim(pg_options_to_table(pg_class.reloptions)::text, '()') as option_kvp -- Convert to text (fillfactor,95), then strip off ( and )

 from pg_class
 join pg_namespace
   on pg_namespace.oid = pg_class.relnamespace

where pg_namespace.nspname = $1
  and relname              = $2
  and reloptions    is not null
),

unpacked_options AS (
select $1                             as schema_name,
       $2                             as table_name,
       split_part(option_kvp, ',', 1) as option,
       split_part(option_kvp, ',', 2) as value

    from packed_options
)

select * from unpacked_options;

$BODY$;
4

1 回答 1

0

数字表明您的策略不起作用,并且绝大多数更新都不热。您还说明了原因:即使您将索引列更新为原始值,您也不会获得 HOT 更新。

UPDATE解决方案是通过在语句中仅包含真正修改的索引列来进行区分。

95 的Afillfactor值也相当高,除非您的表的行数非常小。使用 90 或 85 等设置可能会获得更好的结果。

于 2021-11-06T05:22:11.797 回答