我正在尝试一些代码,当我只有约 50 个请求要发送时效果很好,但现在我正在尝试更多请求(大约 200 个),我觉得服务器可能会在 60 个请求后关闭连接,因为我得到了“无”响应。
这是我正在使用的代码:
urls = []
for kw in keywords:
urls.append("https://play.google.com/store/search?q=" + kw + "&c=apps&hl=fr")
from lxml import html
rs = (grequests.get(u) for u in urls)
responses = grequests.map(rs)
for idx, r in enumerate(responses): # r = results page
time.sleep(2)
responses_tree = html.fromstring(r.content) # Readable HTML
回复:
Out[893]:
[<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
<Response [200]>,
None,
<Response [200]>,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
None]
和回溯:
AttributeError Traceback (most recent call last)
C:\Users\Nathan\googlestore.py in <module>()
249 for idx, r in enumerate(responses): # r = results page
250 time.sleep(2)
--> 251 responses_tree = html.fromstring(r.content) # Readable HTML
252 apps_links = responses_tree.xpath('//a[@class="title"]/@href') # Get all links
253 if len(apps_links) > 10:
AttributeError: 'NoneType' object has no attribute 'content'
我怎样才能解决这个问题?我应该在发送 50 个请求后等待 30 秒,然后再发送 50 个吗?有人有关于向其服务器发出的请求的 Google 文档的链接吗?
谢谢!