在过去的几个月里,googleBot 一直在访问我的网站 [index.php] 上不再存在的文件,因为所有以正确语言到正确主页的路由都是通过 htaccess 中的 apache 重写规则处理的。
Ans to,我注释掉了我的 .htaccessDirectoryIndex index.php
规则
RewriteEngine on
RewriteBase /
Options +FollowSymLinks -Indexes -ExecCGI
# DirectoryIndex index.php (not needed anymore, index.php doesnt exist)
# DirectoryIndex /en/home (should it be set to this now??)
目前,一切都很好:http://website.org
根目录立即重定向到/en/home
通过 301 永久重定向!
但
66.249.67.142 / == crawl-66-249-67-142.googlebot.com 一次又一次地访问我的网站,试图读取不存在的 index.php。我该怎么办??
潜入带有此类条目的无尽错误日志文件(可怜的 googlebot 我认为它可能更智能......)
[Fri Mar 04 20:48:30 2011] [error] [client 66.249.66.177] File does not exist:
/var/www/vhosts/site.com/httpdocs/index.php
[Fri Mar 04 20:58:59 2011] [error] [client 66.249.66.177] File does not exist:
/var/www/vhosts/site.com/httpdocs/index.php
[Fri Mar 04 21:00:18 2011] [error] [client 66.249.67.142] File does not exist:
/var/www/vhosts/site.com/httpdocs/index.php
[Fri Mar 04 21:01:05 2011] [error] [client 66.249.66.177] File does not exist:
/var/www/vhosts/site.com/httpdocs/index.php
[Fri Mar 04 21:12:28 2011] [error] [client 66.249.66.164] File does not exist:
/var/www/vhosts/site.com/httpdocs/index.php
[Fri Mar 04 21:27:30 2011] [error] [client 66.249.68.115] File does not exist:
/var/www/vhosts/site.com/httpdocs/index.php