我在 AWS Ubuntu 服务器上设置了 scrapy 和 scrapy-splash。它可以正常工作一段时间,但几个小时后我会开始收到这样的错误消息;
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.5/site-
packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/home/ubuntu/.local/lib/python3.5/site-
packages/twisted/python/failure.py", line 393, in throwExceptionIntoGe
nerator
return g.throw(self.type, self.value, self.tb)
File "/home/ubuntu/.local/lib/python3.5/site-
packages/scrapy/core/downloader/middleware.py", line 43, in process_re
quest
defer.returnValue((yield download_func(request=request,spider=spider)))
twisted.internet.error.ConnectionRefusedError: Connection was refused by
other side: 111: Connection refused.
我会发现 docker 中的启动过程要么已终止,要么没有响应。
我一直在运行启动过程;
sudo docker run -p 8050:8050 scrapinghub/splash
根据scrapy-splash指令。
我尝试在 tmux shell 中启动该进程以确保 ssh 连接不会干扰 splah 进程,但没有运气。
想法?