这只是与我使用下面的 grok 过滤器剖析要可视化到 kibana 的数据的帖子有关,下面是我在我的 logstash conf 文件中使用的内容,并根据需要处理数据,但今天我进入了它没有根据需要过滤数据的情况。
Kibana 的正确视觉效果如下:
接收时间:2019 年 2 月 1 日,21:00:04.105 浮点数:0.5,0.0类型:rmlog主机名:dba-foxon93 日期:19/02/01用户_1:dv_vxehw @version:1 Hour_since:06命令:rm -rf /data/rg /日志
logstash conf 文件中的 grok 过滤器:
match => { "message" => "%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3} %{GREEDYDATA:Command}" }
我的logstash conf文件:
input {
file {
path => [ "/data/mylogs/*.txt" ]
start_position => beginning
sincedb_path => "/dev/null"
type => "tac"
}
}
filter {
if [type] == "tac" {
grok {
match => { "message" => "%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3} %{GREEDYDATA:Command}" }
add_field => [ "received_at", "%{@timestamp}" ]
remove_field => [ "@version", "host", "message", "_type", "_index", "_score" ]
}
}
}
output {
if [type] == "rmlog" {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "tac-%{+YYYY.MM.dd}"
}
}
}
下面是正在处理的新数据,但我没有得到该数据的主机名、命令等字段。
dbproj01,19/02/01,00:04,23-hrs,cvial,cvial 120804 0.0 0.0 106096 1200 pts/90 S Jan30 0:00 /bin/sh -c /bin/rm -f ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.cxt ../../../../../../
tools.lnx86/dfII/etc/context/64bit/hBrowser.toc ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.aux ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.ini ; (CUR_DIR=`pwd` ;
cd ../../../../obj/linux-x86-64/optimize/bin/virtuoso ; ${CUR_DIR}/../../../../../../tools.lnx86/dfII/bin/virtuoso -ilLoadIL hBrowserBuildContext.il -log hBrowserBuildContext.log -nograph && [ `/bi
n/grep -c Error hBrowserBuildContext.log` = 0 ]) || (echo '*** Error: Failed to build hBrowser context.' ; /bin/rm -f ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.cxt ../../../../..
/../tools.lnx86/dfII/etc/context/64bit/hBrowser.toc ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.aux ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.ini ; exit 1),/pro
j/cvial/WS/BUNGEE/REBASE_190120-138_2/tools.lnx86/dfII/group/bin/src