我需要您的帮助来优化查询。一个表是一个日志表,它有数百万个条目,我试图将我的查询打破到 < 1s。我的查询应该给出一个总体概述,因此应该很快。我确信我可以在脚本的帮助下对列表进行多个简单的查询。但是脚本可以做什么,我认为mysql可以做什么-至少我希望如此。也许并不是所有的部分都得到了最好的使用,但我被困在一个查询中,它产生了一个临时表和文件排序(我发现这真的很糟糕)。在阅读时,我发现使用了一些整洁且放置良好的索引,但现在我被困在一个特定的点上。
让我向您展示我的最终查询及其结果:
SELECT
ps.SERVER_ID,
ps.FULLNAME,
SUM(CASE WHEN pml.ID_TYPE = 3 THEN 1 ELSE 0 END) 'amount_warning',
SUM(CASE WHEN pml.ID_TYPE = 4 THEN 1 ELSE 0 END) 'amount_error',
SUM(CASE WHEN pml.ID_TYPE = 5 THEN 1 ELSE 0 END) 'amount_alert',
SUM(CASE WHEN pml.ID_TYPE = 7 THEN 1 ELSE 0 END) 'amount_critical'
FROM
PAR_SERVER ps
INNER JOIN
PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID
WHERE
pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY )
GROUP BY
ps.SERVER_ID;
这是我得到的:
mysql> [thequeryabove]
[...]
59 rows in set (11.69 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014447 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
这是我当前的表设置:
mysql> describe PAR_SERVER;
+----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+----------------+
| SERVER_ID | int(255) | NO | PRI | NULL | auto_increment |
| FULLNAME | varchar(255) | YES | | NULL | |
| SHORTNAME | varchar(255) | YES | MUL | NULL | |
+----------------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
mysql> show indexes from PAR_SERVER;
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_SERVER | 0 | PRIMARY | 1 | SERVER_ID | A | 142 | NULL | NULL | | BTREE | |
| PAR_SERVER | 1 | shortname | 1 | SHORTNAME | A | 142 | NULL | NULL | YES | BTREE | |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)
mysql> select count(*) from PAR_SERVER;
+----------+
| count(*) |
+----------+
| 142 |
+----------+
1 row in set (0.00 sec)
mysql> describe PAR_MONITORINGv2_LOG;
+--------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+----------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| ID_TYPE | int(11) | NO | MUL | NULL | |
| ID_SERVICE | int(11) | NO | MUL | NULL | |
| SERVER_ID | int(11) | NO | MUL | NULL | |
| MESSAGE | tinytext | NO | | NULL | |
| CREATED_DATE | datetime | NO | | NULL | |
+--------------+----------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
mysql> show indexes from PAR_MONITORINGv2_LOG;
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_MONITORINGv2_LOG | 0 | PRIMARY | 1 | ID | A | 3998188 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_TYPE | 1 | ID_TYPE | A | 7 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | ID_SERVICE | 1 | ID_SERVICE | A | 5 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 1 | SERVER_ID | A | 66 | NULL | NULL | | BTREE | |
| PAR_MONITORINGv2_LOG | 1 | SERVER_ID-ID_TYPE | 2 | ID_TYPE | A | 258 | NULL | NULL | | BTREE | |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
6 rows in set (0.00 sec)
mysql> select count(*) from PAR_MONITORINGv2_LOG;
+----------+
| count(*) |
+----------+
| 3998386 |
+----------+
1 row in set (0.00 sec)
这是逐步分解我的查询的时间结果。在花了这么长时间修复每个部分之后,我可能会一步一步地走。但是目前只有运行时间为 2.30 秒的查询对这个问题感兴趣。
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID WHERE pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY ) GROUP BY ps.SERVER_ID;
[...]
59 rows in set (6.41 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | ALL | SERVER_ID,SERVER_ID-ID_TYPE | NULL | NULL | NULL | 4014788 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID GROUP BY ps.SERVER_ID;
[...]
59 rows in set (2.30 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | pml | index | SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID | 4 | NULL | 4015694 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | ps | eq_ref | PRIMARY | PRIMARY | 4 | database.pml.SERVER_ID | 1 | |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT pml.SERVER_ID FROM PAR_MONITORINGv2_LOG pml GROUP BY pml.SERVER_ID;
[...]
65 rows in set (0.00 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| 1 | SIMPLE | pml | range | NULL | SERVER_ID | 4 | NULL | 67 | Using index for group-by |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)
我可以通过为 (SERVER_ID, ID_TYPE) 定义一个索引来改进查询,正如我的以下示例查询所证实的那样:
mysql> SELECT count(*) 'count_warnings' FROM PAR_MONITORINGv2_LOG pml WHERE pml.SERVER_ID = 191 AND pml.ID_TYPE = 3 GROUP BY pml.SERVER_ID;
[...]
1 row in set (0.01 sec)
mysql> explain [thequeryabove]
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| 1 | SIMPLE | pml | ref | ID_TYPE,SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID-ID_TYPE | 8 | const,const | 10254 | Using index |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
1 row in set (0.00 sec)
我现在被困在最崩溃的查询中,执行时间长达 2.30 秒。我不知道如何为没有任何 where 子句的此类查询使用索引。