3

当用户访问我们的网站时，我有一个 Postgres 表，其中包含Segment.com 创建的anonymous_id（字符串）和（日期时间）列。timestamp

有~5M 行，~1M 不同的anonymous_id

我想查询每月发现的不同匿名 ID 的数量。

到目前为止我有这个，它有效，但在 PSequel 中超时（我可以运行它多次并限制日期）

SELECT count(1), "month"
FROM (
    SELECT DISTINCT anonymous_id, 
    date_trunc('month', "timestamp") as "month"
    FROM pages
    -- WHERE "timestamp" between '2018-01-01' and '2018-02-01'
) as dt
GROUP BY 2
ORDER BY 2

我在anonymous_id 和timestamp 上有一个索引

结果EXPLAIN ANALYSE

                                                                 QUERY PLAN                                                                  
---------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=1667977.72..1667978.22 rows=200 width=8) (actual time=115861.803..115861.807 rows=27 loops=1)
   Sort Key: (date_trunc('month'::text, pages."timestamp"))
   Sort Method: quicksort  Memory: 26kB
   ->  HashAggregate  (cost=1667968.07..1667970.07 rows=200 width=8) (actual time=115861.763..115861.766 rows=27 loops=1)
         Group Key: (date_trunc('month'::text, pages."timestamp"))
         ->  Unique  (cost=1554502.82..1592324.57 rows=5042900 width=45) (actual time=97492.062..115468.396 rows=1158934 loops=1)
               ->  Sort  (cost=1554502.82..1567110.07 rows=5042900 width=45) (actual time=97492.060..113983.496 rows=5042900 loops=1)
                     Sort Key: pages.anonymous_id, (date_trunc('month'::text, pages."timestamp"))
                     Sort Method: external merge  Disk: 285936kB
                     ->  Seq Scan on pages  (cost=0.00..682820.25 rows=5042900 width=45) (actual time=0.088..25601.944 rows=5042900 loops=1)
 Planning time: 10.335 ms
 Execution time: 115910.353 ms
(12 rows)

当前指数（包括下面 Thorsten Kettner 建议的组合指数）

Indexes:
    "pages_pkey" PRIMARY KEY, btree (id)
    "idx_anonymous_id" btree (anonymous_id)
    "idx_date_trunc_anon_id" btree (date_trunc('month'::text, timezone('UTC'::text, "timestamp")), anonymous_id)
    "idx_path" btree (path)
    "idx_timestamp" btree ("timestamp")
    "idx_url" btree (url)
    "idx_user_id" btree (user_id)
    "pages_activity_type_idx" btree (activity_type)
4

2 回答 2

1

我能想到的唯一一件事就是摆脱derived table，因为你不需要它：

SELECT count(distinct anonymous_id), date_trunc('month', "timestamp") AS "month"
FROM pages
GROUP BY date_trunc('month', "timestamp")
ORDER BY date_trunc('month', "timestamp");
于 2018-02-02T14:09:39.323 回答
0

group by您想要一个以您的表达式开头的索引。理想情况下，它还包含anonymous_id：

CREATE INDEX idx1 ON pages( date_trunc('month', "timestamp") , anonymous_id);

如果你想要一个WHERE子句，那么你首先需要它的标准：

CREATE INDEX idx2 ON pages( "timestamp" , date_trunc('month', "timestamp") , anonymous_id);

正如 Greg Viers 所示，您的查询比需要的更复杂。我会和他的一起去。两个查询所需的索引相同。

于 2018-02-02T14:23:44.943 回答