2

我将带有 Logstash 的 Glassfish 4 日志文件发送到 ElasticSearch 接收器。如何使用 Logstash 从消息字段中删除尾随换行符？

我的活动如下所示：

{
  "@timestamp" => "2013-11-21T13:29:33.081Z",
  "message" => "[2013-11-21T13:29:32.577+0000] [glassfish 4.0] [INFO] [] [javax.resourceadapter.mqjmsra.lifecycle] [tid: _ThreadID=142 _ThreadName=Thread-43] [timeMillis: 1385040572577] [levelValue: 800] [[\n  MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter stopped.]]\n",
  "@version" => "1",
  "tags" => ["multiline", "date_filtered"],
  "host" => "myhost",
  "path" => "../server.log"
}
4

2 回答 2

11

第二种解决方案是使用Logstash 的 mutate 过滤器。它允许您剥离字段的值。

filter {
  # Remove leading and trailing whitspaces (including newline etc. etc.)
  mutate {
    strip => "message"
  }
}
于 2013-12-03T14:43:15.273 回答
2

您必须使用具有正确模式的多行过滤器来告诉logstash，每行带有前置空格的行都属于之前的行。将此行添加到您的 conf 文件中。

filter{
  ...
  multiline {
    type => "gflogs"
    pattern => "\[\#\|\d{4}"
    negate => true
    what => "previous"
  }
  ...
}

您还可以包含 grok 插件来处理时间戳和过滤来自beeing索引的不规则行。

在同一台机器上查看具有单个 logstash 实例的完整堆栈

input {
  stdin {
    type => "stdin-type"
  }

  file {
    path => "/path/to/glassfish/logs/*.log"
    type => "gflogs"
  }
}

filter{
  multiline {
    type => "gflogs"
    pattern => "\[\#\|\d{4}"
    negate => true
    what => "previous"
  }

  grok {
    type => "gflogs"
    pattern => "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:loglevel}\|%{DATA:server_version}\|%{JAVACLASS:category}\|%{DATA:kv}\|%{DATA:message}\|\#\]"
    named_captures_only => true
    singles => true
  }

  date {
    type => "gflogs"
    match => [ "timestamp", "ISO8601" ]
  }

  kv {
    type => "gflogs"
    exclude_tags => "_grokparsefailure"
    source => "kv"
    field_split => ";"
    value_split => "="
  }
}

output {
  stdout { codec => rubydebug }
  elasticsearch { embedded => true }
}

这对我有用。请在logstash-usergroup上查看这篇文章。我还可以建议伟大的和最新的logstash 书。这也是支持logstash作者工作的好方法。

希望在任何 JUG-Berlin 活动中见到您！

于 2013-12-03T10:21:43.060 回答