3

I've had my cURL implementation running successfully for the last few months without hiccups; however, last week I suddenly started to have a problem with one specific website (www.viewmag.com). I can visit the site (and have it resolve) perfectly in a browser, but cURL returns the following: 

* About to connect() to www.viewmag.com port 80 (#0)
*   Trying 205.178.145.65... * Timeout
* connect() timed out!
* Closing connection #0

For sanity, I tried to ping the website with two different boxes, but each ping timed out.

Box 1 (Linux):

ping www.viewmag.com
PING www.viewmag.com (205.178.145.65) 56(84) bytes of data.

Box 2 (Windows):

ping www.viewmag.com

Pinging www.viewmag.com [205.178.145.65] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

My cURL is as follows:

$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, 'http://www.viewmag.com');
curl_setopt ($ch, CURLOPT_USERAGENT, 'cURL crawler');
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt ($ch, CURLOPT_AUTOREFERER, true);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 2);
$html = curl_exec($ch);

Does anyone have any thoughts as to why cURL is failing and why I would be able to visit this site in a browser, but not be able to ping/cURL it? Thanks in advance

4

2 回答 2

7

  1. 也许您的服务器 IP 在该站点上被禁止？

  2. 也许尝试设置更长的超时时间？我访问了那个网站，它运行的很慢，你可能需要超过 5 秒。

后来补充：

看起来你的服务器IP被禁止了。

我试过这个（它的代码副本，更改在评论中）：

<?php

$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, 'http://www.viewmag.com');

// I changed UA here
curl_setopt ($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1');

curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt ($ch, CURLOPT_AUTOREFERER, true);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 2);
$html = curl_exec($ch);

// I added this 
echo $html; 

?>

它适用于我的测试服务器（德国数据中心）。

于 2013-04-30T16:48:04.507 回答
1

他们很可能提高了服务器的安全性。服务器中的某些设置已更改以阻止您卷曲它。尝试伪装成已知的用户代理。Pinging 可能不起作用，因为他们刚刚关闭了 ping 服务器，因此可以阻止分布式拒绝服务 (DDOS) 等攻击。可悲的是，目前无法确定哪种确切的组合可以或将使其发挥作用。您将需要反复试验。

于 2013-04-30T16:45:19.137 回答