0

我正在努力通过用and替换implicitly_wait来加速 Selenium 网络抓取。我对如何实现这一目标有点困惑。WebDriverWaitsend_keysclick

这是我的代码inplicitly_wait：

def ncd_web_scraping(df):
    df['new_column'] = 'Not_sure'
    url = 'url'
    for i in df.index:
        driver = webdriver.Chrome()
        driver.implicitly_wait(5)
        driver.get(url)
        name = driver.find_element_by_xpath('//*[@id="person"]')
        name.send_keys(df.loc[i, 'Name'])
        state = driver.find_element_by_xpath('//*[@id="state"]')
        state.send_keys(df.loc[i, 'State'])
        botton = driver.find_element_by_xpath('/html/body/div[2]/form/button')
        botton.click()

        soup = BeautifulSoup(driver.page_source, 'html.parser')
        if soup.find('h5'):
            df.loc[i, 'new_column'] = 'Yes'
        else:
            df.loc[i, 'new_column'] = 'No'


    return df

谁能帮我webDriveWait改写代码？

我感谢您的帮助。

4

1 回答 1

0

用WebDriverWait替换您的有效代码块将是：

def ncd_web_scraping(df):
    df['new_column'] = 'Not_sure'
    url = 'url'
    for i in df.index:
        driver = webdriver.Chrome()
        driver.get(url)
        WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='person']"))).send_keys(df.loc[i, 'Name'])
        driver.find_element_by_xpath('//*[@id="state"]').send_keys(df.loc[i, 'State'])
        driver.find_element_by_xpath('/html/body/div[2]/form/button').click()
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        if soup.find('h5'):
            df.loc[i, 'new_column'] = 'Yes'
        else:
            df.loc[i, 'new_column'] = 'No'
    return df

注意：您必须添加以下导入：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
于 2020-01-13T16:05:55.357 回答