0

我无法使用任何 Bellow 方法访问该站点，$url 地址在我所有的浏览器中都可以使用，但是，我无法从该站点获取数据....这怎么可能？除了在浏览器中之外，甚至无法获取robots.txt（' https://www.natterer-modellbau.de/robots.txt ）......

我在 Google 上看到该网站的结果，当我不能访问该网站时，Google 如何访问该网站？

该页面在第一次尝试时拒绝了我的爬虫，他们不能已经阻止我的服务器 IP 了吗？我的脚本可以访问所有其他 URL —— 我很沮丧 :) 请帮助...

$url = 'https://www.natterer-modellbau.de/Flugzeuge';

$pageHeaders = get_headers($url,1); // DOES NOT WORK - TIMES OUT
file_get_contents($url); // DOES NOT WORK -Times OUT 

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$res = curl_exec($ch);
$rescode = curl_getinfo($ch, CURLINFO_HTTP_CODE); 
curl_close($ch) ;
echo $res; // DOES NOT WORK - TIMES OUT
4

1 回答 1

0

为我工作。您可能在代理后面。

php > $url = 'https://www.natterer-modellbau.de/Flugzeuge';
php >
php > $pageHeaders = get_headers($url,1); // DOES NOT WORK - TIMES OUT
php > file_get_contents($url); // DOES NOT WORK -Times OUT
php >
php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_URL, $url);
php > curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
php > curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
php > curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
php > $res = curl_exec($ch);
php > $rescode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
php > curl_close($ch) ;
php > echo $res;
<!DOCTYPE html>
<html lang="de">
<head>

        <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
        <meta name="description" content="Elektro, Segler/E-Segler, Verbrenner">
        <meta name="keywords" content="Elektro, Segler/E-Segler, Verbrenner">

....省略了许多行

终于结束了html

        </script>

    <script>
        jtl.load(["asset/plugin_js_head?v=4.05","asset/jtl3.js?v=4.05","asset/plugin_js_body?v=4.05",]);
            </script>

</body>
</html>
php >
于 2018-03-15T23:29:33.663 回答