2

我有一个硒项目。我将在 selenium 中使用 Crawlera 代理。我已经有一个 Crawlera 的 API Key。

        headless_proxy = "127.0.0.1:3128"
        proxy = Proxy({
            'proxyType': ProxyType.MANUAL,
            'httpProxy': headless_proxy,
            'ftpProxy' : headless_proxy,
            'sslProxy' : headless_proxy,
            'noProxy'  : ''
        })
        user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) ' \
                     'Chrome/80.0.3987.132 Safari/537.36'
        chrome_option = webdriver.ChromeOptions()
        chrome_option.add_argument('--no-sandbox')
        chrome_option.add_argument('--disable-dev-shm-usage')
        chrome_option.add_argument('--ignore-certificate-errors')
        chrome_option.add_argument("--disable-blink-features=AutomationControlled")
        chrome_option.add_argument(
            'user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) '
            'Chrome/80.0.3987.132 Safari/537.36')
        chrome_option.headless = True
        prefs = {"profile.managed_default_content_settings.images": 2}
        chrome_option.add_experimental_option("prefs", prefs)

        capabilities = dict(DesiredCapabilities.CHROME)
        proxy.add_to_capabilities(capabilities)

        driver = webdriver.Chrome(desired_capabilities=capabilities, options=chrome_option)
        driver.set_page_load_timeout(600)
        #driver = webdriver.Chrome(options=chrome_option)

那么如何设置 API KEY 呢？然后我想在 Scrapinghub 上部署代码。如何将 Crawlera 应用到 selenium 中，以便它在 Scrapy 云上正常工作？请帮我。谢谢。

4

0 回答 0