1

我编写了一个 python 函数来根据一些参数（一系列单词）对网站进行评分。该函数使用 Python Mechanize，并且在大多数情况下都能正常工作。

但是，对于某些网站，它会一直挂在那里，直到我在终端上按 ctrl+c。我猜这是某种与javascript相关的问题，有没有办法围绕这个构建超时函数？

这是我的功能：

def rateSite(site_url,comparisonWords):
   #open the site
   localBrowser = mechanize.Browser()
   localBrowser.addheaders = [('User-agent', 'Mozilla/5.1 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/9.0.1')]
   localBrowser.set_handle_robots(False)
   site = localBrowser.open(site_url,timeout=5000)
   html = site.read()

   #rate the site
   for i in comparisonWords.split():
      #do some rating math

   return rating

这是我在 ctrl+c 上得到的回溯：

site=localBrowser.open(site_url,timeout=5000)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 612, in http_response
    "http", request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 219, in error
    result = apply(self._call_chain, args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 146, in http_error_302
    return self.parent.open(new)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 612, in http_response
    "http", request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 219, in error
    result = apply(self._call_chain, args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 146, in http_error_302
    return self.parent.open(new)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 612, in http_response
    "http", request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 219, in error
    result = apply(self._call_chain, args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 146, in http_error_302
    return self.parent.open(new)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 578, in http_response
    self._sleep(pause)
KeyboardInterrupt

任何有关如何解决此问题或为其建立超时的帮助将不胜感激。

谢谢！

4

1 回答 1

1

timeout=5000超过一小时；你可能是说timeout=5。

默认情况mechanize下，在放弃之前最多遵循 10 次重定向，请参阅HTTPRedirectHandler.max_redirections.

于 2012-03-13T15:51:55.053 回答