python - 如何从命令行使用 Scrapy 传递表单数据？

Question

如何从命令行传递用户名和密码？谢谢！

class LoginSpider(Spider):
    name = 'example.com'
    start_urls = ['http://www.example.com/users/login.php']

    def parse(self, response):
        return [FormRequest.from_response(response,
                    formdata={'username': 'john', 'password': 'secret'},
                    callback=self.after_login)]

    def after_login(self, response):
        # check login succeed before going on
        if "authentication failed" in response.body:
            self.log("Login failed", level=log.ERROR)
            return

        # continue scraping with authenticated session...

Answer 1

打开终端并确保已安装scrapy。

1. scrapy shell

2. from scrapy.http import FormRequest

3. request=FormRequest(url='http://www.example.com/users/login.php',formdata={'username': 'john','password':'secret',})

信息：

• 刮痧 1.0.0

Answer 2

你可以做

scrapy crawl spidername -a username="john" -a password="secret"

接着

class LoginSpider(Spider):
    name = 'example.com'
    start_urls = ['http://www.example.com/users/login.php']

    def parse(self, response):
        return [FormRequest.from_response(response,
                    formdata={'username': self.username, 'password': self.password},
                    callback=self.after_login)]

    def after_login(self, response):
        # check login succeed before going on
        if "authentication failed" in response.body:
            self.log("Login failed", level=log.ERROR)
            return

        # continue scraping with authenticated session...