1

我在 filebeat.yml 中使用 Filebeat 多行模式，它从单个文件中获取输入，如下所示：

2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

文件节拍.yml

multiline:
pattern: Identifier
negate: true
match: after

我使用上面的配置来匹配行中的“标识符”。输出应如所愿

event -1 :
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

event -2 :
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

4

1 回答 1

0

根据您的示例输入，看起来我们可以使用包含的行requestStartIdentifier: Identifier来表示新事件的开始。我使用https://play.golang.org/p/BZ2ujeOZZ-来测试不同的多行参数。

文件节拍配置：

filebeat:
  prospectors:
    - input_type: log
      paths:
        - input.txt
      multiline:
        pattern: 'requestStartIdentifier: Identifier$'
        negate:  true
        match:   after

output:
  console:
    pretty: true

Filebeat 输出（扩展了换行符）：

{
  "@timestamp": "2016-10-06T21:51:27.244Z",
  "beat": {
    "hostname": "host",
    "name": "host"
  },
  "input_type": "log",
  "message": "2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
    ContentType: text/xml; charset=utf-8
    ContextPath: 
    LocalAddr: 
    LocalName: 
    PathInfo: 
    PathTranslated: 
    QueryString: 
    RequestURI: 
    RequestURL: 
    RemoteHost: 
    ServletPath: 
    Header: Host: 
    Header: Content-Length: 
    Header: Accept-Encoding: 
    Header: SOAPAction: \"\"
    Header: User-Agent: Apache-HttpClient/4.2.1 
    Header: Content-Type: text/xml; charset=utf-8
    Header: Connection: Keep-Alive
    Header: Accept: text/xml

2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
\u003cenv:Envelope\u003e\u003c/env:Envelope\u003e
",
  "offset": 962,
  "source": "input.txt",
  "type": "log"
}
{
  "@timestamp": "2016-10-06T21:51:27.244Z",
  "beat": {
    "hostname": "host",
    "name": "host"
  },
  "input_type": "log",
  "message": "2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
    ContentType: text/xml; charset=utf-8
    ContextPath: 
    LocalAddr: 
    LocalName: 
    PathInfo: 
    PathTranslated: 
    QueryString: 
    RequestURI: 
    RequestURL: 
    RemoteHost: 
    ServletPath: 
    Header: Host: 
    Header: Content-Length: 
    Header: Accept-Encoding: 
    Header: SOAPAction: \"\"
    Header: User-Agent: Apache-HttpClient/4.2.1 
    Header: Content-Type: text/xml; charset=utf-8
    Header: Connection: Keep-Alive
    Header: Accept: text/xml

2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
\u003cenv:Envelope\u003e\u003c/env:Envelope\u003e",
  "offset": 1923,
  "source": "input.txt",
  "type": "log"
}
于 2016-10-06T14:44:48.533 回答