-1

我正在尝试从 tomcat 日志中提取错误|异常。由于 tomcat 日志在多行中有错误详细信息。我想将每个新日志条目视为我的记录，即日期可以作为记录分隔符。

Oct 4, 2012 4:00:38 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already.  Could not load com.google.common.base.Stopwatch.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)
        at com.myweb.ontest.stats.RpcInterceptor.intercept(RpcInterceptor.java:45)
        at com.myweb.ontest.platform.SupplySource$Iface$$EnhancerByCGLIB$$6b5e8142.finalize()
        at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
        at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
        at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
Oct 4, 2012 4:00:38 PM Org.apache.catalina.loader ....

请让我知道如何使用 awk/perl 脚本来完成。

谢谢

4

3 回答 3

1

我首先在记录之间添加一个空行：

awk '/^[A-Z][a-z]/ && f++{print ""}1' file

因此您可以通过使用空行作为记录分隔符在后续的 awk 脚本中轻松处理它：

awk '/^[A-Z][a-z]/ && f++{print ""}1' file |
awk -v RS= -F'\n' '{print "Record #" NR; for (i=1;i<=NF;i++) print "Field #" i, "[" $i "]"}'
Record #1
Field #1 [Oct 4, 2012 4:00:38 PM org.apache.catalina.loader.WebappClassLoader loadClass]
Field #2 [INFO: Illegal access: this web application instance has been stopped already.  Could not load com.google.common.base.Stopwatch.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.]
Field #3 [java.lang.IllegalStateException]
Field #4 [        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)]
Field #5 [        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)]
Field #6 [        at com.myweb.ontest.stats.RpcInterceptor.intercept(RpcInterceptor.java:45)]
Field #7 [        at com.myweb.ontest.platform.SupplySource$Iface$$EnhancerByCGLIB$$6b5e8142.finalize()]
Field #8 [        at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)]
Field #9 [        at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)]
Field #10 [        at java.lang.ref.Finalizer.access$100(Finalizer.java:14)]
Field #11 [        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)]
Record #2
Field #1 [Oct 7, 2012 4:00:38 PM org.apache.catalina.loader.WebappClassLoader loadClass]
Field #2 [INFO: just a dummy record]
Field #3 [java.lang.IllegalStateException]
Field #4 [        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)]
Field #5 [        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)]
Field #6 [        at com.myweb.ontest.stats.RpcInterceptor.intercept(RpcInterceptor.java:45)]
Field #7 [        at com.myweb.ontest.platform.SupplySource$Iface$$EnhancerByCGLIB$$6b5e8142.finalize()]
Field #8 [        at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)]
Field #9 [        at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)]
Field #10 [        at java.lang.ref.Finalizer.access$100(Finalizer.java:14)]
Field #11 [        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)]

您没有发布您希望输出的样子，因此如果上述内容没有为您提供足够的信息来执行此操作，请发布您的示例输入的预期输出

于 2012-12-02T16:26:11.383 回答
1

假设您要将这些错误放入 MySQL 数据库中，我会这样做。

#!/usr/bin/perl -Tw

use strict;
use warnings;
use English qw( -no_match_vars $OS_ERROR );
use Readonly;
use DBI;

Readonly my $LOG_FILENAME => '/var/log/tomcat.log';

Readonly my $DATE_REGEX => qr{
    \w{3} \s \d+, \s        # Oct 4,
    \d{4}         \s        # 2012
    \d+:\d+:\d+   \s \w{2}  # 4:00:38 PM
}xms;

my $sth;
{
    my $dbh = DBI->connect(
        'DBI:mysql:database=errors;host=localhost',
        'error_monitor',
        '*********',
        { 'RaiseError' => 1 }
    );

    die $DBI::errstr
        if !$dbh;

    $sth = $dbh->prepare_cached( q{
        INSERT INTO error (
            date,
            text
        ) VALUES (
            ?,
            ?
        )
    } );

    die $DBI::errstr
        if !$sth;
}

my $fh;
{
    open $fh, '<', $LOG_FILENAME
        or die "open $LOG_FILENAME: $OS_ERROR";
}

my ( $count, %record ) = ( 0, () );

while ( my $line = <$fh> ) {

    if ( $line =~ m{\A ( $DATE_REGEX ) \s ( .+ ) }xms ) {

        my ( $date, $text ) = ( $1, $2 );

        if (%record) {
            $sth->execute( @record{qw( date text )} );
            %record = ();
            $count++;
        }

        @record{qw( date text )} = ( $date, $text );
    }
    elsif ( exists $record{date} ) {

        $record{text} .= $line;
    }
    else {

        warn "malformed message: $line";
    }
}

if (%record) {
    $sth->execute( @record{qw( date text )} );
    $count++;
}

close $fh
    or die "close $LOG_FILENAME: $OS_ERROR";

print "inserted $count error records\n"
    or die "print: $OS_ERROR";

__END__
于 2012-12-02T21:52:33.373 回答
0

在 perl 中你可以这样做：-

use strict;
my $log = join('', <STDIN>);
my %entries = split(/([\r\n]*\w{3} \d+, \d+ \d+:\d+:\d+ [AP]M )/m, $log);

foreach my $key(keys %entries)
{
    print "$key\n";
    print "$entries{$key}\n";
}

上面的代码从标准输入读取整个日志，将其拆分为日期并存储日期，输入哈希并在最后打印哈希。

于 2012-12-02T16:32:26.730 回答