我正在尝试通过scrapy下载图像。这是我的不同文件:
项目.py
class DmozItem(Item):
title = Field()
image_urls = Field()
images = Field()
设置.py
BOT_NAME = 'tutorial'
SPIDER_MODULES = ['tutorial.spiders']
NEWSPIDER_MODULE = 'tutorial.spiders'
ITEM_PIPELINES = ['scrapy.contrib.pipeline.images.ImagesPipeline']
IMAGES= '/home/mayank/Desktop/sc/tutorial/tutorial'
蜘蛛
class DmozSpider(BaseSpider):
name = "wikipedia"
allowed_domains = ["wikipedia.org"]
start_urls = [
"http://en.wikipedia.org/wiki/Pune"
]
def parse(self, response):
hxs = HtmlXPathSelector(response)
items = []
images=hxs.select('//a[@class="image"]')
for image in images:
item = DmozItem()
link=image.select('@href').extract()[0]
link = 'http://en.wikipedia.com'+link
item['image_urls']=link
items.append(item)
尽管进行了所有这些设置,我的管道没有被激活。请帮忙。我是这个框架的新手。