-3

I got a huge log in zipped files, i need to write out lines by specific data and if the line with the same contains XML message with the same sessionID will be written to the file to.

The log structure:

 2013-08-16 16:31:06,810 ( 122:            rogate) [98839276727]  INFO  -      UId:10453, GId:5422: new CONX started, Application Context: disconnected
 2013-08-16 16:31:34,210 ( 122:            rogate) [98839276727]  INFO  -      UId:32453, GId:1213: new CONX started, Application Context: disconnected
 2013-08-16 16:31:45,110 ( 122:            rogate) [98839276727]  INFO  -      UId:11453, GId:2133: new CONX started, Application Context: disconnected
 2013-08-16 16:31:45,729 (1093:               jms_con.cpp) [140561430333184] DEBUG  - Received XML TextMessage:
 <?xml version="1.0" encoding="UTF-8"?><>
 <version>1</version>
 <sessionId>114532133</sessionId>
 <networkProtocolId>CAPv2</networkProtocolId>
 <trafficType>Forwarding</trafficType>
  <messages>
   <reportNotificationAck/>
 <superviseReq>
 <requestSequenceNr>0</requestSequenceNr>
 <time>60000</time>
 <releaseAfterTimeExpires>false</releaseAfterTimeExpires>
  <playWarningTone>false</playWarningTone>
 </superviseReq>
 <eventReportReq>
 <requestSequenceNr>1</requestSequenceNr>
 <events>
<routeSelectFailure monitorMode="Interrupt"/>
<busy monitorMode="Interrupt"/>
<noAnswer monitorMode="Interrupt">
  <noAnswerTimer>180000</noAnswerTimer>
</noAnswer>
<answer monitorMode="Notify"/>
<disconnectCalling monitorMode="Interrupt"/>
<disconnectCalled monitorMode="Interrupt"/>
<abandon monitorMode="Notify"/>
</events>
</eventReportReq>
<continueProcessing>
<requestSequenceNr>2</requestSequenceNr>
<moreEventsExpected>true</moreEventsExpected>
<interruptEventReceived>true</interruptEventReceived>
</continueProcessing>
2013-08-16 16:59:03,666 (1252:            capgw_main.cpp) [140561430333184]  INFO  - UId:57371, GId:7137: STAT_ISIG_PROCESSING: 0.001007.
2013-08-16 16:59:03,666 ( 888:  tcap_context_storage.cpp) [140561430333184] DEBUG  - UId:57371, GId:7137: updating the Last Appl. Access Time.
2013-08-16 16:59:03,666 ( 937:  tcap_context_storage.cpp) [140561430333184] DEBUG  - UId:57371, GId:7137: new Appl. message has different direction as previously stored one, calculating the response time.
2013-08-16 16:59:03,666 (1260:            capgw_main.cpp) [140561430333184] DEBUG  - UId:57371, GId:7137: TCAP Context Storage updated successfully (received iSig message).
2013-08-16 16:59:03,666 (1263:            capgw_main.cpp) [140561430333184]  INFO  - UId:57371, GId:7137: STAT_ISIG_RESP_TIME: 0.023346
2013-08-16 16:59:03,666 ( 767:  tcap_context_storage.cpp) [140561430333184] DEBUG  - UId:57371, GId:7137: updating the Last TCAP Access Time.

After the third line an XML message present with same sessionID as the line UiD+GiD. I need to write this lines to a new files, like this:

 2013-08-16 16:31:45,110 ( 122:            rogate) [98839276727]  INFO  -      UId:11453, GId:2133: new CONX started, Application Context: disconnected
 2013-08-16 16:31:45,729 (1093:               jms_con.cpp) [140561430333184] DEBUG  - Received XML TextMessage:
 <?xml version="1.0" encoding="UTF-8"?><>
 <version>1</version>
 <sessionId>114532133</sessionId>
 <networkProtocolId>CAPv2</networkProtocolId>
 <trafficType>Forwarding</trafficType>
  <messages>
   <reportNotificationAck/>
 <superviseReq>
 <requestSequenceNr>0</requestSequenceNr>
 <time>60000</time>
 <releaseAfterTimeExpires>false</releaseAfterTimeExpires>
  <playWarningTone>false</playWarningTone>
 </superviseReq>
 <eventReportReq>
 <requestSequenceNr>1</requestSequenceNr>
 <events>
<routeSelectFailure monitorMode="Interrupt"/>
<busy monitorMode="Interrupt"/>
<noAnswer monitorMode="Interrupt">
  <noAnswerTimer>180000</noAnswerTimer>
</noAnswer>
<answer monitorMode="Notify"/>
<disconnectCalling monitorMode="Interrupt"/>
<disconnectCalled monitorMode="Interrupt"/>
<abandon monitorMode="Notify"/>
</events>
</eventReportReq>
<continueProcessing>
<requestSequenceNr>2</requestSequenceNr>
<moreEventsExpected>true</moreEventsExpected>
<interruptEventReceived>true</interruptEventReceived>
</continueProcessing>

Where a file named as XML message sessionID, like 114532133_something.txt and write this every two log messages into a new file.

Thanks for helping!

Edit:

Trying to do in a script with not so many sucess.

#!/usr/bin/awk -f

BEGIN { FS=":|," }
FNR==NR && /INFO/ {
    a[$0,$1,$2,$3,$4,$5,$6,$7,$8,$9,$10]++ ;
    next

}

END

{
    for (i in a) print i
}
4

1 回答 1

0

我把它分成了多个任务。可能有人可以将这一切结合在一起。

1- 查找所有包含 INFO 的 Session ID 表单行并将其存储到文件t1

awk -F":|," 'FNR==NR && /INFO/ {a[$6$8]++;next} END {for (i in a) print i }' xml_file >t1

2- 删除所有日志行（从 2013 年开始）并将其存储在t2

awk  '!/^2013/' xml_file >t2

3- 打印每个 XML 批量，如果它确实包含在步骤 1 中找到的 ID 之一

awk 'FNR==NR {a[$1]++;next} FNR==1 {RS="</continueProcessing>"} { for (i in a) {if ($0~i) print}}'  t1 t2

我假设所有 XML 部分都以 . 如果这不是真的，这将不起作用。

于 2013-08-21T11:11:28.267 回答