0

I'm using Logstash to forward error logs from app servers to ES. Everything is working fine except that log timestamp going as string to ES.

Here is my log format

[Date:2015-03-25 01:29:09,554] [ThreadId:4432] [HostName:AEPLWEB1] [Host:(null)] [ClientIP:(null)] [Browser:(null)] [UserAgent:(null)] [PhysicalPath:(null)] [Url:(null)] [QueryString:(null)] [Referrer:(null)] [Carwale.Notifications.ExceptionHandler] System.InvalidCastException: Unable to cast object of type 'Carwale.Entity.CMS.Articles.ArticleDetails' to type 'Carwale.Entity.CMS.Articles.ArticlePageDetails'. at Carwale.Cache.Core.MemcacheManager.GetFromCacheCore[T](String key, TimeSpan cacheDuration, Func`1 dbCallback, Boolean& isKeyFirstTimeCreated)

Filter configuration for logstash forwarder

filter {  

    multiline {
            pattern => "^\[Date:%{TIMESTAMP_ISO8601}"
            negate => true
            what => "previous"
        }   

    grok {
        match => [ "message", "(?:Date:%{TIMESTAMP_ISO8601:log_timestamp})\] \[(?:ThreadId:%{NUMBER:ThreadId})\] \[(?:HostName:%{WORD:HostName})\] \[(?:Host:\(%{WORD:Host})\)\] \[(?:ClientIP:\(%{WORD:ClientIP})\)\] \[(?:Browser:\(%{WORD:Browser})\)\] \[(?:UserAgent:\(%{WORD:UserAgent})\)\] \[(?:PhysicalPath:\(%{WORD:PhysicalPath})\)\] \[(?:Url:\(%{WORD:Url})\)\] \[(?:QueryString:\(%{WORD:QueryString})\)\] \[(?:Referrer:\(%{WORD:Referrer})\)\] \[%{DATA:Logger}\] %{GREEDYDATA:err_message}" ]  
    }

    date {
        match => [ "log_timestamp", "MMM dd YYY HH:mm:ss","MMM  d YYY HH:mm:ss", "ISO8601" ]
        target => "log_timestamp"
    }

    mutate {
        convert => ["ThreadId", "integer"]
    }
}

How I can make it date in ES? Please help. Thanks in advance.

4

1 回答 1

0

我有类似的问题。现在使用以下解决方法修复它。

      grok {
        match => {
            "message" => "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}[T ]%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}" 

             }
        }

  grok{
    match => {
        "second" => "(?<asecond>(^[^,]*))"  }
    }

mutate {
        add_field => { 
        "timestamp" => "%{year}-%{month}-%{day} %{hour}:%{minute}:%{asecond}"
                             }
    }

    date{  match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ] timezone=> "UTC" target => "log_timestamp" }

谢谢，

于 2015-07-22T12:15:57.703 回答