3

我正在使用 grequests 对 crunchbase API 执行多个请求。

不幸的是，他们没有提供根据公司更新时间查询数据的方法，所以为了获得最近资助的公司，我必须刮掉他们所有的公司名称，然后对每个公司执行单独的请求。

尝试多处理后，我找到了模块 grequest。代码如下：

import grequests

def query():
    names = [list of company names] #around 200 per chunk
    base_url = 'http://api.crunchbase.com/v/1/company/%s.js?api_key=%s' % ('%s',API_KEY)
    rs = (grequests.get(u) for u in urls)
    chunk_responses = grequests.imap(rs, size = 5)

但是，我似乎收到以下错误：

Traceback (most recent call last):
File "/home/user/anaconda/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(*self.args, **self.kwargs)
File "/home/user/anaconda/lib/python2.7/site-packages/grequests.py", line 128, in send
return r.send(stream=stream)
File "/home/user/anaconda/lib/python2.7/site-packages/grequests.py", line 71, in send
self.url, **merged_kwargs)
File "/home/user/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 383, in request
resp = self.send(prep, **send_kwargs)
File "/home/user/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 506, in send
history = [resp for resp in gen] if allow_redirects else []

File "/home/user/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 168, in           
resolve_redirects
allow_redirects=False,
File "/home/user/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "/home/user/anaconda/lib/python2.7/site-packages/requests/adapters.py", line 382, in send
raise ConnectionError(e)

ConnectionError: HTTPConnectionPool(host='ec2-107-21-104-179.compute-1.amazonaws.com',   port=80): 
Max retries exceeded with url: /v/1/company/spiceworks.js (Caused by <class 'socket.gaierror'>:     
[Errno -2] Name or service not known)
<Greenlet at 0x47dd870: send(<grequests.AsyncRequest object at 0x63d1c50>)> failed with    
ConnectionError

知道为什么我会收到此错误吗？

4

0 回答 0