故事是这样的,我们有一个名为 syslog 的 MySQL 数据库,有以下几个表:
Tables_in_syslog |
+-----------------------------+
| plugin_camm_syslog |
| plugin_camm_syslog_MonthDec |
| plugin_camm_syslog_MonthJan |
| plugin_camm_syslog_incoming
表plugin_camm_syslog_incoming
有存储引擎 MEMORY,其他的是MyISAM
.
这是每月第一天从 cron 运行的脚本:
<?php
# This script rotates the mysql syslog-ng database
# on a monthly basis. Place in "/etc/cron.monthly".
# To change to weekly/daily, adjust "$date" and place
# in the appropriate cron folder.
$link = mysql_connect("localhost", "syslogadmin", "sysadmin");
if (!$link) {
die("Could not connect: " . mysql_error());
}
echo "Connected successfully\n";
// make syslog the current db
$db_selected = mysql_select_db("syslog", $link);
if (!$db_selected) {
die ("Can\"t use syslog : " . mysql_error());
}
// Rename the table, appending short month
$last= mktime(0, 0, 0, date("m")-1, date("d"), date("Y"));
$drop= mktime(0, 0, 0, date("m")-3, date("d"), date("Y"));
$d=date("M", $last); $dd=date("M", $drop);
$date="Month$d"; $dropdate="Month$dd";
// drop old table
// drop 2 month old table
$result=mysql_query("DROP TABLE `plugin_camm_syslog_$dropdate`");
//$result=mysql_query("DROP TABLE `plugin_camm_syslog_$date`");
//make new date
$last= mktime(0, 0, 0, date("m")-1, date("d"), date("Y")); $d=date("M", $last);
$date="Month$d";
#echo "XX $date\n";
#die ("test $result:".mysql_error());
$result = mysql_query("ALTER TABLE `plugin_camm_syslog` RENAME `plugin_camm_syslog_$date`");
if (!$result) { die("Invalid query: " . mysql_error()); }
// Recreate the original table
$result="
CREATE TABLE `plugin_camm_syslog` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`host` varchar(128) DEFAULT NULL,
`sourceip` varchar(45) NOT NULL,
`facility` varchar(10) DEFAULT NULL,
`priority` varchar(10) DEFAULT NULL,
`sys_date` datetime DEFAULT NULL,
`message` text,
`status` tinyint(4) NOT NULL DEFAULT '0',
`alert` smallint(3) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`,`sys_date` ),
KEY `facility` (`facility`),
KEY `priority` (`priority`),
KEY `sourceip` (`sourceip`),
KEY `status` (`status`),
KEY `alert` (`alert`),
KEY `status_date` (`status`,`sys_date`),
KEY `sys_date` (`sys_date`)
) ENGINE=MyISAM
DEFAULT CHARSET=latin1
COMMENT='camm plugin SYSLOG Data'
PARTITION BY RANGE ( dayofyear(sys_date)) (
PARTITION syslog_day0 VALUES LESS THAN (0) ENGINE = MyISAM,
PARTITION syslog_day14 VALUES LESS THAN (14) ENGINE = MyISAM,
PARTITION syslog_day28 VALUES LESS THAN (28) ENGINE = MyISAM,
PARTITION syslog_day42 VALUES LESS THAN (42) ENGINE = MyISAM,
PARTITION syslog_day56 VALUES LESS THAN (56) ENGINE = MyISAM,
PARTITION syslog_day70 VALUES LESS THAN (70) ENGINE = MyISAM,
PARTITION syslog_day84 VALUES LESS THAN (84) ENGINE = MyISAM,
PARTITION syslog_day98 VALUES LESS THAN (98) ENGINE = MyISAM,
PARTITION syslog_day112 VALUES LESS THAN (112) ENGINE = MyISAM,
PARTITION syslog_day126 VALUES LESS THAN (126) ENGINE = MyISAM,
PARTITION syslog_day140 VALUES LESS THAN (140) ENGINE = MyISAM,
PARTITION syslog_day154 VALUES LESS THAN (154) ENGINE = MyISAM,
PARTITION syslog_day168 VALUES LESS THAN (168) ENGINE = MyISAM,
PARTITION syslog_day182 VALUES LESS THAN (182) ENGINE = MyISAM,
PARTITION syslog_day196 VALUES LESS THAN (196) ENGINE = MyISAM,
PARTITION syslog_day210 VALUES LESS THAN (210) ENGINE = MyISAM,
PARTITION syslog_day224 VALUES LESS THAN (224) ENGINE = MyISAM,
PARTITION syslog_day238 VALUES LESS THAN (238) ENGINE = MyISAM,
PARTITION syslog_day252 VALUES LESS THAN (252) ENGINE = MyISAM,
PARTITION syslog_day266 VALUES LESS THAN (266) ENGINE = MyISAM,
PARTITION syslog_day280 VALUES LESS THAN (280) ENGINE = MyISAM,
PARTITION syslog_day294 VALUES LESS THAN (294) ENGINE = MyISAM,
PARTITION syslog_day308 VALUES LESS THAN (308) ENGINE = MyISAM,
PARTITION syslog_day322 VALUES LESS THAN (322) ENGINE = MyISAM,
PARTITION syslog_day336 VALUES LESS THAN (336) ENGINE = MyISAM,
PARTITION syslog_day350 VALUES LESS THAN (350) ENGINE = MyISAM,
PARTITION syslog_day364 VALUES LESS THAN (364) ENGINE = MyISAM,
PARTITION syslog_day369 VALUES LESS THAN (369) ENGINE = MyISAM
)
";
if (mysql_query($result)) {
echo "success in plugin-camm table creation.\n";
}
else {
echo "no table created.\n";
}
if ( mysql_query($result)) {
echo "success in table creation.\n";
} else {
echo "no table created.\n";
}
mysql_close($link);
$link = mysql_connect("localhost", "cacxt", "cacxt");
if (!$link) {
die("Could not connect: " . mysql_error());
}
echo "Connected successfully\n";
$db_selected = mysql_select_db("cacti", $link);
if (!$db_selected) {
die ("Can\"t use cacti : " . mysql_error());
}
$result=mysql_query("TRUNCATE TABLE `plugin_camm_tree2`");
$result=mysql_query("TRUNCATE TABLE `plugin_camm_temp`");
$result=mysql_query("TRUNCATE TABLE `plugin_camm_keys`");
$result=mysql_query("UPDATE `plugin_camm_rule` SET `actual_triggered` = '0';");
mysql_close($link);
?>
问题是服务器无法再处理这么多的数据,并且每个月大约有 20 天服务器面临性能问题。我不是 MySQL 忍者,但试图了解发生了什么,似乎索引填充太大 [服务器可以处理的更大]
看:
:/var/lib/mysql/syslog# ls -lh | grep G total 21G
-rw-rw---- 1 mysql mysql 2,8G 2012-12-15 00:50 plugin_camm_syslog_MonthDec#P#syslog_day350.MYD
-rw-rw---- 1 mysql mysql 1,5G 2013-01-31 14:12 plugin_camm_syslog_MonthDec#P#syslog_day350.MYI
-rw-rw---- 1 mysql mysql 1,6G 2012-12-29 00:46 plugin_camm_syslog_MonthDec#P#syslog_day364.MYD
-rw-rw---- 1 mysql mysql 2,0G 2013-01-14 00:46 plugin_camm_syslog_MonthJan#P#syslog_day14.MYD
-rw-rw---- 1 mysql mysql 1,1G 2013-01-28 08:01 plugin_camm_syslog_MonthJan#P#syslog_day14.MYI
-rw-rw---- 1 mysql mysql 2,1G 2013-01-28 05:41 plugin_camm_syslog_MonthJan#P#syslog_day28.MYD
-rw-rw---- 1 mysql mysql 1,1G 2013-01-28 08:01 plugin_camm_syslog_MonthJan#P#syslog_day28.MYI
-rw-rw---- 1 mysql mysql 1,7G 2013-02-11 00:46 plugin_camm_syslog#P#syslog_day42.MYD
-rw-rw---- 1 mysql mysql 1,7G 2013-02-21 07:29 plugin_camm_syslog#P#syslog_day56.MYD
今天我需要运行脚本来轮换系统日志,我的问题是它会保留我 1 月 / 2 月的数据吗?
或者有人可以建议如何减少索引使用?