proxylist无法加载Scrapy云

Question

模块即时通讯使用“https://github.com/aivarsk/scrapy-proxies” 设置如指定和个人电脑工作完美，如果是链接到PC上现有的txt磁贴。

我在settings.py文件中尝试了几种不同的Scrapy Cloud方法。

我在同一个文件夹中添加文件“proxylist.txt”比低于除了该项目的设置我把它上传到“https://dl.dropboxusercontent.com/s/esdm19mnvz2yguf/proxylist.txt”

我的替代名称： PROXY_LIST =“https://dl.dropboxusercontent.com/s/esdm19mnvz2yguf/proxylist.txt” 或 PROXY_LIST = 'proxylist.txt' 或 PROXY_LIST = '/proxylist.txt' PROXY_LIST = '../proxylist.txt'

，如果我不喜欢它在我的电脑PROXY_LIST = 'proxylist.txt'，它的功能就像一个魅力，但不是我一次加载Scrapy Cloud。

我收到错误。

Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks result = g.send(result) File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 90, in crawl six.reraise(*exc_info) File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 72, in crawl self.engine = self._create_engine() File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 97, in _create_engine return ExecutionEngine(self, lambda _: self.stop()) File "/usr/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 69, in init self.downloader = downloader_cls(crawler) File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/init.py", line 88, in init self.middleware = DownloaderMiddlewareManager.from_crawler(crawler) File "/usr/local/lib/python2.7/site-packages/scrapy/middleware.py", line 58, in from_crawler return cls.from_settings(crawler.settings, crawler) File "/usr/local/lib/python2.7/site-packages/scrapy/middleware.py", line 36, in from_settings mw = mwcls.from_crawler(crawler) File "/app/python/lib/python2.7/site-packages/scrapy_proxies/randomproxy.py", line 55, in from_crawler return cls(crawler.settings) File "/app/python/lib/python2.7/site-packages/scrapy_proxies/randomproxy.py", line 35, in init fin = open(self.proxy_list) IOError: [Errno 2] No such file or directory: '../proxylist.txt'

请我需要一些帮助。

Answer 1

你很可能不在您的setup.py说明中包含该文件。

提供此功能的机制是MANIFEST.in文件。这是相对比较简单：MANIFEST.in真的只是一个指定的文件或水珠相对文件路径的列表，包括：

include README.rst 
include docs/*.txt 
include funniest/data.json 

为了使在安装时包的文件夹站点包内被复制这些文件，您需要为setup()函数提供include_package_data=True。

请参阅http://python-packaging.readthedocs.io/en/latest/non-code-files.html