1

我正在从 .txt 加载数据以进行抓取。但是，URL 要求我分解该变量并对其执行 +/- 2。例如，如果值为 2342，我需要为 URL 创建 2340 和 2344。

我猜测如何分解它：

 $args{birth_year} = ($args{birth_year} - 2) . '-' . ($args{birth_year} + 2);

然后如何将其放入 URL 中？

这是代码的相关部分：

  use strict;
  use warnings;
  use WWW::Mechanize::Firefox;
  use Data::Dumper;
  use LWP::UserAgent;
   use JSON;
  use CGI qw/escape/;
  use HTML::DOM;

  open(my $l, 'locations2.txt') or die "Can't open locations: $!";

 while (my $line = <$l>) {
    chomp $line;
     my %args;
     @args{qw/givenname surname birth_place birth_year gender race/} = split /,/, $line;
     $args{birth_year} = ($args{birth_year} - 2) . '-' . ($args{birth_year} + 2);
      my $mech = WWW::Mechanize::Firefox->new(create => 1, activate => 1);
     $mech->get("https://familysearch.org/search/collection/index#count=20&query=%2Bgivenname%3A$args{givenname}20%2Bsurname%3A$args{surname}20%2Bbirth_place%3A$args{birth_place}%20%2Bbirth_year%3A1910-1914~%20%2Bgender%3A$args{gender}20%2Brace%3A$args{race}&collection_id=2000219");

例如

输入是：

Benjamin,Schuvlein,Germany,1912,M,White

所需的网址是：

https://familysearch.org/search/collection/index#count=20&query=%2Bgivenname%3ABenjamin%20%2Bsurname%3ASchuvlein%20%2Bbirth_place%3AGermany%20%2Bbirth_year%3A1910-1914~%20%2Bgender%3AM% 20%2Brace%3AWhite&collection_id=2000219

4

3 回答 3

3

为什么你不能改变这一行：

$mech->get("https://familysearch.org/search/collection/index#count=20&query=%2Bgivenname%3A$args{givenname}20%2Bsurname%3A$args{surname}20%2Bbirth_place%3A$args{birth_place}%20%2Bbirth_year%3A1910-1914~%20%2Bgender%3A$args{gender}20%2Brace%3A$args{race}&collection_id=2000219");

对此：

$mech->get("https://familysearch.org/search/collection/index#count=20&query=%2Bgivenname%3A$args{givenname}20%2Bsurname%3A$args{surname}20%2Bbirth_place%3A$args{birth_place}%20%2Bbirth_year%3A$args(birth_year)~%20%2Bgender%3A$args{gender}20%2Brace%3A$args{race}&collection_id=2000219");

注意：我改变了这一点：

%3A1910-1914~%20

对此：

%3A$arg(birth_year)~%20
于 2013-02-12T02:24:09.837 回答
0

One way to do it:

file content:
link1
link2
...
linkn

use Data::Dumper;
use strict;
use warnings;

local $/=undef;
open(FILE,'<',$filename) or die $filename;
my $i = 1;
while (my $line = <FILE>){
  chomp($line);
  print "line: $line\n";
  my $tempfile = './$i.html';$i++;
  $mech->get( $line, ':content_file' => $tempfile, synchronize => 1 );
}
于 2013-02-11T08:23:06.647 回答
0

这个答案没有考虑输入中的数据是否需要进行 URL 编码，即如果姓氏是“von Schtupp”，它需要变成“von%20Schtupp”

我没有对此进行测试，因此可能存在拼写错误或小错误。尽管如此，这是我会使用的方法。我的回答还假设您不在乎搜索条件的出现顺序。

my %query_params = (
    givenname => $args{givenname},
    surname   => $args{surname},
    birth_place => $args{birth_place},
    birth_year => sprintf("%d-%d", $args{birth_year} - 2, $args{birth_year} + 2),
    gender     => $args{gender},
    race       => $args{race},
);
my $query_parameter = join '%20',
                      map { "%2B$_%3A$query_params{$_}" }
                      keys %query_params;
my $url = "https//familysearch.org/search/collection/index#count=20&query=" .
          $query_parameter . "&collection_id=2000219";
于 2013-02-12T01:43:43.523 回答