1

我正在尝试一些代码，当我只有约 50 个请求要发送时效果很好，但现在我正在尝试更多请求（大约 200 个），我觉得服务器可能会在 60 个请求后关闭连接，因为我得到了“无”响应。

这是我正在使用的代码：

urls = []
for kw in keywords:
    urls.append("https://play.google.com/store/search?q=" + kw + "&c=apps&hl=fr")

from lxml import html

rs = (grequests.get(u) for u in urls)
responses = grequests.map(rs)

for idx, r in enumerate(responses): # r = results page
    time.sleep(2)
    responses_tree = html.fromstring(r.content) # Readable HTML

回复：

Out[893]: 
[<Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 None,
 <Response [200]>,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None]

和回溯：

AttributeError                            Traceback (most recent call last)
C:\Users\Nathan\googlestore.py in <module>()
    249 for idx, r in enumerate(responses): # r = results page
    250     time.sleep(2)
--> 251     responses_tree = html.fromstring(r.content) # Readable HTML
    252     apps_links = responses_tree.xpath('//a[@class="title"]/@href') # Get all links
    253     if len(apps_links) > 10:

AttributeError: 'NoneType' object has no attribute 'content'

我怎样才能解决这个问题？我应该在发送 50 个请求后等待 30 秒，然后再发送 50 个吗？有人有关于向其服务器发出的请求的 Google 文档的链接吗？

谢谢！

4

0 回答 0