我scrapy shell myurl
从项目的顶级目录运行。以下是日志输出:
$ scrapy shell "http://www.aufeminin.com/forum/show0_3/grossesse.html"
2013-11-07 12:02:08+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: auf)
2013-11-07 12:02:08+0100 [scrapy] DEBUG: Enabled extensions: TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, RedirectMiddleware, CookiesMiddleware, HttpCompressionMiddleware, ChunkedTransferMiddleware, DownloaderStats
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Enabled item pipelines:
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Telnet console listening on 0.0.0.0:6023
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Web service listening on 0.0.0.0:6080
2013-11-07 12:02:09+0100 [AufemininKeywordClassifierSpider] INFO: Spider opened
2013-11-07 12:02:09+0100 [AufemininKeywordClassifierSpider] DEBUG: Crawled (200) <GET http://www.aufeminin.com/forum/show0_3/grossesse.html> (referer: None)
[s] Available Scrapy objects:
[s] hxs <HtmlXPathSelector xpath=None data=u'<html>\r\n<head>\r\n<title>Forum Grossesse -'>
[s] item {}
[s] request <GET http://www.aufeminin.com/forum/show0_3/grossesse.html>
[s] response <200 http://www.aufeminin.com/forum/show0_3/grossesse.html>
[s] settings <CrawlerSettings module=<module 'auf.settings' from '/home/louist/Desktop/auf/auf/settings.pyc'>>
[s] spider <AufKeywordSpider 'AufemininKeywordClassifierSpider' at 0x3d41ed0>
[s] Useful shortcuts:
[s] shelp() Shell help (print this help)
[s] fetch(req_or_url) Fetch request (or URL) and update local objects
[s] view(response) View response in a browser
本教程建议我应该有一个名为的变量sel
,其中包含一个选择器。为什么不是这样?