python - Scrapyd：如何指定已部署项目可以使用的库和公共文件夹？

Question

Scrapyd 是我们可以部署项目的服务。但是我面临一个问题。我有一个名为 MyScrapers 的项目，其蜘蛛类使用如下导入语句：

from mylibs.common.my_base_spider import MyBaseSpider

my_base_spider 的路径是/home/myprojectset/mylibs/common/my_base_spider

在设置环境变量PYTHONPATH=$HOME/myprojectset/时，我可以使用 scrapy 命令运行 MyScrapers：scrapy crawl MyScrapers。

但是当我使用 scrapyd 通过以下命令部署 MyScrapers 时：scrapy deploy scrapyd2 -p MyScrapers，我收到以下错误：

服务器响应 (200): {"status": "error", "message": "ImportError: No module named mylibs.common.my_base_spider"}

请告诉如何使部署的项目使用这些库？

Answer 1

You need to edit your setup.py and/or MANIFEST.in to declare that it has a mylibs package dependency.

See the distutils documentation for more information.

Answer 2

I found the answer by adding mylibs to site-packages of python by using setup.py inside mylib folder. That way I could import everything inside mylib in my projects. Actually mylibs were way outside from the location where setup.py of my deploy-able project is present. setup.py looks for packages on same level and inside the folders where it is located.