我使用带有 OAuth 身份验证的 cpan 库 AnyEvent::Twitter::Stream 做了一个推文爬虫,但我遇到了一些问题。有时,Twitter 会停止通过流 API 发送推文。然后,我让我的程序休眠一段时间,然后它应该返回以收集更多推文,但这并没有发生。该脚本只是继续尝试重新连接,但无法建立任何连接。会是什么呢?我的代码如下所示:
my $done = AnyEvent->condvar;
my $count=0;
# track keywords através da OAuth
my $guard = AnyEvent::Twitter::Stream->new(
consumer_key => "...", #My Oauth authentication enters here
consumer_secret => "...",
token => "...",
token_secret => "...",
method => "filter",
track => $track,
on_tweet => \&got_tweet,
on_error => \&connection_close,
timeout => 45,
);
sub connection_close{
my($headers)=@_;
print "HEADERS: $headers\n\n\n\n\n";
open(FOUT,">>arquivoalerta.txt");
my $Agora = time();
my $HoraLocal = localtime($Agora);
my @Tempo = split(/ +/,$HoraLocal);
print FOUT "Parei de coletar às @Tempo Streaming API";
if ($count==0){
print FOUT "Dormindo por 10 segundos\n";
print "Dormindo por 10 segundos\n";
sleep 10;
}
elsif($count==1){
print FOUT "Dormindo por 20 segundos\n";
print "Dormindo por 20 segundos\n";
sleep 20;
}
else{
print FOUT "Dormindo por 240 segundos\n";
print "Dormindo por 240 segundos\n";
sleep 240;
}
close(FOUT);
$count++;
warn "Connection to Twitter closed";
}
sub got_tweet {
#Here I treat the data and store it in a database ...
}
$done->recv;
感谢大家,
蒂亚戈