6

希望有人可以帮助我！

我有一个关于logstash的问题。我成功地了解了以下日期：26/Jun/2013:14:00:26 +0200

接下来，我希望将此日期用作事件的@timestamp。如您所知，logstash 会自动添加时间戳。

替换logstash正在添加的时间戳可以通过日期过滤器来完成。我添加了以下日期过滤器： match => [ "date", "dd/MMM/YYYY:HH:mm:ss Z"]

但是，由于某种原因，这行不通。当我测试它时，我看到 logstash 只是添加了他自己的时间戳。

代码：

grok {
    type => "log-date"
    pattern => "%{HTTPDATE:date}"
}

date{
    type => "log-date"
    match => [ "date", "dd/MMM/YYYY:HH:mm:ss Z"]
}

我需要这样做，所以我可以将事件添加到 elasticsearch。

提前致谢！

4

2 回答 2

8

我使用了以下方法：

# strip the timestamp and force event timestamp to be the same.
# the original string is saved in field %{log_timestamp}.
# the original logstash input timestamp is saved in field %{event_timestamp}.
grok {
  patterns_dir => "./patterns"
  match => [ "message", "%{IRODS_TIMESTAMP:log_timestamp}" ]
  add_tag => "got_syslog_timestamp"
  add_field => [ "event_timestamp", "%{@timestamp}" ]
}

date {
  match => [ "log_timestamp", "MMM dd HH:mm:ss" ]
}

mutate {
        replace => [ "@timestamp", "%{log_timestamp}" ]
}

My problem now is that, even if @timestamp is replaced, I would like to convert it to a ISO8601-compatible format first so that other programs don't have problems interpreting it, like the timestamp present in "event_timestamp":

     "@timestamp" => "Mar  5 14:38:40",
       "@version" => "1",
           "type" => "irods.relog",
           "host" => "ids-dev",
           "path" => "/root/logstash/reLog.2013.03.01",
            "pid" => "5229",
          "level" => "NOTICE",
  "log_timestamp" => "Mar  5 14:38:40",
"event_timestamp" => "2013-09-17 12:20:28 UTC",
           "tags" => [
    [0] "got_syslog_timestamp"
]

You could convert it easily since you have the year information... In my case I would have to parse it out of the "path" (filename) attribute... but still, there does not seem to be an convert_to_iso8901 => @timestamp directive.

Hope this helps with your issue anyway! :)

于 2013-09-17T12:43:36.800 回答
5

The above answer is just a work around !, try to add locale => "en" to your code.
If not added, the date weekdays and month names will be parsed with the default platform locale language (spanish, french or whatever) and that's why it didn't work (since your log is in english).

date{
    type => "log-date"
    match => [ "date", "dd/MMM/YYYY:HH:mm:ss Z"]
    locale => "en"
}
于 2014-03-06T15:50:57.670 回答