0

I'm getting occasional AttributeErrors with code of the following sort. I set a mechanize instance up with:

self.mech = mechanize.Browser(factory=mechanize.RobustFactory())
self.cj = mechanize.CookieJar()
self.mech.set_cookiejar(self.cj)
self.mech.set_proxies({'http': <snipped>})
self.mech.set_handle_robots(False)

USER_AGENT = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
headers = [h for h in self.mech.addheaders if h[0].lower() != 'user-agent']
headers.append(('User-agent', USER_AGENT))
self.mech.addheaders = headers

And I use it as such:

resp = self.mech.open(the_url)
html = resp.read()
resp.close()

The latter snippet occasionally raises exceptions:

...
    html = resp.read()
AttributeError: 'NoneType' object has no attribute 'read'

In other cases, the traceback is actually:

...
    resp.close()
  File "C:\Python26\lib\site-packages\mechanize\_response.py", line 88, in close
    self.wrapped.close()
  File "C:\Python26\lib\site-packages\mechanize\_response.py", line 368, in close
    wrapped.close()
  File "C:\Python26\lib\socket.py", line 273, in close
    self._sock.close()
AttributeError: 'NoneType' object has no attribute 'close'

That is, the .read() does not fail, but the .close() does. More tracebacks:

...
    html = resp.read()
  File "C:\Python26\lib\site-packages\mechanize\_response.py", line 190, in read
    self.__cache.write(self.wrapped.read())
  File "C:\Python26\lib\socket.py", line 348, in read
    data = self._sock.recv(rbufsize)
  File "C:\Python26\lib\httplib.py", line 542, in read
    s = self.fp.read(amt)
  File "C:\Python26\lib\socket.py", line 377, in read
    data = self._sock.recv(left)
error: [Errno 10035] A non-blocking socket operation could not be completed immediately

And:

    resp = self.mech.open(the_url)
  File "C:\Python26\lib\site-packages\mechanize\_mechanize.py", line 203, in open
    return self._mech_open(url, data, timeout=timeout)
  File "C:\Python26\lib\site-packages\mechanize\_mechanize.py", line 249, in _mech_open
    self._set_response(response, False)
  File "C:\Python26\lib\site-packages\mechanize\_mechanize.py", line 308, in     _set_response
    self._factory.set_response(response)
  File "C:\Python26\lib\site-packages\mechanize\_html.py", line 623, in set_response
    data = response.read()
  File "C:\Python26\lib\site-packages\mechanize\_response.py", line 190, in read
    self.__cache.write(self.wrapped.read())
  File "C:\Python26\lib\socket.py", line 348, in read
    data = self._sock.recv(rbufsize)
  File "C:\Python26\lib\httplib.py", line 542, in read
    s = self.fp.read(amt)
  File "C:\Python26\lib\socket.py", line 377, in read
    data = self._sock.recv(left)
AttributeError: 'NoneType' object has no attribute 'recv'

Why might this happen? The mechanize documentation isn't very good, and a cursor poke-through of the source reveals that it is relatively convoluted.

4

0 回答 0