2

我有以下文字：

GET /mac/_base_v1/images/chrome/background_repeat.jpg HTTP/1.1  
Host: www.microsoft.com  
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20100101 Firefox/13.0  
Accept: image/png,image/*;q=0.8,*/*;q=0.5  
Accept-Language: en-us,en;q=0.5  
Accept-Encoding: gzip, deflate  
Referer: http://www.microsoft.com/mac/base-css  
DNT: 1  
Connection: keep-alive  
HTTP/1.1 200 OK  
Cache-Control: max-age=900  
Content-Type: image/jpegGET /mac/_base_v1/modules/button/images  /buttonlarge_yellownormal.png HTTP/1.1  
Host: www.microsoft.com  
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20100101 Firefox/13.0   
Accept: image/png,image/*;q=0.8,*/*;q=0.5  
Accept-Language: en-us,en;q=0.5  
Accept-Encoding: gzip, deflate  
Referer: http://www.microsoft.com/mac/css  
DNT: 1

和下面的 Perl 正则表达式

while ($1 =~m/((GET|PUT|POST|CONNECT)\s+\S+)(?:(?!GET|PUT|POST|CONNECT\s+\S+).)*?Host:\s([^\n]+).*?User-Agent:\s([^\n]+).*?Referer:\s([^\n]+).*?Connection:/msg) {
    # do something
}

它匹配这个很好

GET /mac/_base_v1/modules/button/images/buttonlarge_yellownormal.png  
www.microsoft.com  
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20100101 Firefox/13.0  
http://www.microsoft.com/mac/css

但是，我还需要它来检查以下文本：

GET /vi/k_dbVP4r4V4/hqdefault.jpg HTTP/1.1  
Host: i.ytimg.com  
User-Agent: Apple iPad v4.3.5 YouTube v1.0.0.8L1  
Accept-Language: en-us, *;q=0.5  
Gdata-Version: 2  
X-Gdata-Client: ytapi-apple-ipad  
Accept: */*  
Accept-Encoding: gzip, deflate  
Connection: keep-alive  
Q2J}

并匹配以下内容：

GET /vi/k_dbVP4r4V4/hqdefault.jpg HTTP/1.1  
i.ytimg.com  
Apple iPad v4.3.5 YouTube v1.0.0.8L1

同时仍然能够匹配成功呈现的先前文本。

4

2 回答 2

2

因此，如果我正确理解您的问题，您需要将 Referrer 标头设为可选。您可以通过在与该标题匹配的正则表达式部分周围添加非捕获括号并在右括号后放置一个问号来做到这一点：

(?:Referer:\s([^\n]+))?

如果任何其他标题是可选的，您可以对它们执行相同的操作。

编辑：在第一个丢失的标头之后数据停止被捕获。

这还不完美，因为如果单个数据文件中有多个 HTTP 请求，它就不起作用，但它应该让你朝着正确的方向前进：

use warnings;
use strict;

my $str = <<'END_OF_STR';
GET /vi/k_dbVP4r4V4/hqdefault.jpg HTTP/1.1
Host: i.ytimg.com
User-Agent: Apple iPad v4.3.5 YouTube v1.0.0.8L1
Accept-Language: en-us, *;q=0.5
Gdata-Version: 2
X-Gdata-Client: ytapi-apple-ipad
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
END_OF_STR

my @lines = split m/[\n]/xms, $str;

# Build the regex to match the HTTP methods we care about.
my @methods = qw(GET PUT POST CONNECT);
my $methods_re = join '|', map { quotemeta $_ } @methods;

# Skip to the first request line and print it.
while ( $lines[0] !~ m/ \A $methods_re /xms ) {
    shift @lines;
}
print "$lines[0]\n";
shift @lines;

# Build the regex to match the headers we care about.
my @headers = qw(Host User-Agent Referer Connection);
my $headers_re = join '|', map { quotemeta $_ } @headers;

# Find the headers that we matched.
for my $line (@lines) {
    if ( $line =~ m/ \A (?:$headers_re):\s*(.*) /xms ) {
        print "$1\n";
    }
}

exit;

我将很快添加另一个更新，它将在单个文件中处理多个 HTTP 请求。

编辑：此解决方案正确打印您正在寻找的值，但它只打印它们。如果您想获取每个特定请求的值，则需要更复杂的东西。

use warnings;
use strict;

my $str = <<'END_OF_STR';
GET /vi/k_dbVP4r4V4/hqdefault.jpg HTTP/1.1
Host: i.ytimg.com
User-Agent: Apple iPad v4.3.5 YouTube v1.0.0.8L1
Accept-Language: en-us, *;q=0.5
Gdata-Version: 2
X-Gdata-Client: ytapi-apple-ipad
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
END_OF_STR

my @lines = split m/[\n]/xms, $str;

# Build the regexes to match the HTTP methods and headers we care about.
my @methods = qw(GET PUT POST CONNECT);
my $methods_re = join '|', map { quotemeta $_ } @methods;
my @headers = qw(Host User-Agent Referer Connection);
my $headers_re = join '|', map { quotemeta $_ } @headers;

for my $line (@lines) {
    if ( $line =~ m/ \A $methods_re /xms ) {
        print "$line\n";
    }
    elsif ( $line =~ m/ \A (?:$headers_re):\s*(.*) /xms ) {
        print "$1\n";
    }
}

exit;
于 2012-07-15T03:25:57.313 回答
2

HTTP 请求和响应标头并不像预期的那样容易解析。例如，以下都是等价的：

Accept-Encoding: gzip, deflate

Accept-Encoding: gzip,
    deflate

Accept-Encoding: gzip
Accept-Encoding: deflate

因此，我建议您使用现有的解析器

use strict;
use warnings;
use feature qw( say );
use HTTP::Request qw( );

my $s = <<'__EOI__';
GET /mac/_base_v1/images/chrome/background_repeat.jpg HTTP/1.1  
Host: www.microsoft.com  
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20100101 Firefox/13.0  
Accept: image/png,image/*;q=0.8,*/*;q=0.5  
Accept-Language: en-us,en;q=0.5  
Accept-Encoding: gzip, deflate  
Referer: http://www.microsoft.com/mac/base-css  
DNT: 1  
Connection: keep-alive  
HTTP/1.1 200 OK  
Cache-Control: max-age=900  
Content-Type: image/jpegGET /mac/_base_v1/modules/button/images  /buttonlarge_yellownormal.png HTTP/1.1  
Host: www.microsoft.com  
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20100101 Firefox/13.0   
Accept: image/png,image/*;q=0.8,*/*;q=0.5  
Accept-Language: en-us,en;q=0.5  
Accept-Encoding: gzip, deflate  
Referer: http://www.microsoft.com/mac/css  
DNT: 1  
__EOI__

my ($raw_req, $raw_resp) = split qr{(?=^HTTP/)}m, $s;
my $req = HTTP::Request->parse($raw_req);
say $req->method;
say $req->url;
say $req->user_agent;
say $req->header('User-Agent');  # Same as previous
于 2012-07-15T05:47:33.553 回答