1

在此处输入图像描述我想报废库存产品的名称和数量，但它返回无。

data_insight = []

for n in range(pagenum):
    pages_url = f"https://www.insight.com/en_US/search.html?qtype=all&q=tp-link&qsrc=k&pq=%7B%22pageSize%22%3A100%2C%22currentPage%22%3A{n+1}%2C%22shownFlag%22%3Afalse%2C%22priceRangeLower%22%3Anull%2C%22priceRangeUpper%22%3Anull%2C%22cmtStandards%22%3Atrue%2C%22categoryId%22%3Anull%2C%22setType%22%3Anull%2C%22setId%22%3Anull%2C%22shared%22%3Anull%2C%22groupId%22%3Anull%2C%22cmtCustomerNumber%22%3Anull%2C%22groupName%22%3Anull%2C%22fromLicense%22%3Atrue%2C%22licenseContractIds%22%3Anull%2C%22assortmentIds%22%3Anull%2C%22controller%22%3Anull%2C%22fromcs%22%3Afalse%2C%22searchTerms%22%3A%7B%22TP-LINK%2520TECHNOLOGY%22%3A%7B%22field%22%3A%22field%22%2C%22value%22%3A%22A-HYBRIS-ManufacturerId~0007045098%22%7D%7D%2C%22sortBy%22%3A%22BestMatch%22%7D"
driver.get(pages_url)
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '[class="prod-section-container"]')))
html = driver.page_source
soup = Soup(html)
soup

    for item in soup.select('[class="prod-section-container"]'):
    data_insight.append({
        'title' : item.find("a", class_="select-prod").text,
        'name' : item.find("p", string="Insight Part"),
        'link' : item.find("a", class_="select-prod")['href'],
        'price' : item.find("span", class_="c-currency__value"),
        'stock' : item.find("p", class_="prod-stock").text
    })
    
df_insight = pd.DataFrame(data_insight)
df_insight.drop_duplicates()
df_insight
4

1 回答 1

0

怎么了？

string您可以搜索字符串而不是标签，但以您的方式查找完全匹配。

怎么修？

你可以去，string=re.compile()但这只是第二好的解决方案。

更好的方法是调整您css selector以获取所有 itemprop 属性：

soup.select('#js-search-product-items [itemprop="itemListElement"]')

例子

...
data_insight = []

for n in range(1):
    pages_url = f"https://www.insight.com/en_US/search.html?qtype=all&q=tp-link&qsrc=k&pq=%7B%22pageSize%22%3A100%2C%22currentPage%22%3A{n+1}%2C%22shownFlag%22%3Afalse%2C%22priceRangeLower%22%3Anull%2C%22priceRangeUpper%22%3Anull%2C%22cmtStandards%22%3Atrue%2C%22categoryId%22%3Anull%2C%22setType%22%3Anull%2C%22setId%22%3Anull%2C%22shared%22%3Anull%2C%22groupId%22%3Anull%2C%22cmtCustomerNumber%22%3Anull%2C%22groupName%22%3Anull%2C%22fromLicense%22%3Atrue%2C%22licenseContractIds%22%3Anull%2C%22assortmentIds%22%3Anull%2C%22controller%22%3Anull%2C%22fromcs%22%3Afalse%2C%22searchTerms%22%3A%7B%22TP-LINK%2520TECHNOLOGY%22%3A%7B%22field%22%3A%22field%22%2C%22value%22%3A%22A-HYBRIS-ManufacturerId~0007045098%22%7D%7D%2C%22sortBy%22%3A%22BestMatch%22%7D"
    driver.get(pages_url)
    WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '[class="prod-section-container"]')))
    html = driver.page_source
    soup = Soup(html)
    soup

    for item in soup.select('#js-search-product-items [itemprop="itemListElement"]'):

        data_insight.append({
            'title' : item.find(attrs={'itemprop':'name'}).text,
            'sku' : item.find(attrs={'itemprop':'sku'}).text,
            'link' : item.find(attrs={'itemprop':'url'}).text,
            'price' : item.find("span", class_="c-currency__value").text,
            'stock' : item.find("p", class_="prod-stock").get_text(strip=True).split(' ')[0]
        })

df_insight = pd.DataFrame(data_insight)
df_insight.drop_duplicates()
df_insight

输出

标题 库存单位 关联 价格 股票
TP-Link TL-WR902AC - 无线路由器 - 802.11a/b/g/n/ac - 桌面 TL-WR902AC https://www.insight.com/en_US/shop/product/TL-WR902AC/TP-LINK%20TECHNOLOGY/TL-WR902AC/TPLink-TLWR902AC--wireless-router--80211abgnac--desktop/ 37.99 美元 751
TP-Link Omada ER605 V1 - 路由器 - 台式机 ER605 https://www.insight.com/en_US/shop/product/ER605/TP-LINK%20TECHNOLOGY/ER605/TPLink-Omada-ER605-V1--router--desktop/ 59.99 美元 839
TP-Link TL-PA4010 KIT - 电力线适配器套件 - 壁挂式 TL-PA4010KIT https://www.insight.com/en_US/shop/product/TL-PA4010KIT/TP-LINK%20TECHNOLOGY/TL-PA4010KIT/TPLink-TLPA4010-KIT--powerline-adapter-kit--wallpluggable/ 46.99 美元 442
TP-Link Powerline AV2000 - 入门套件 - 电力线适配器套件 - 可插入墙壁 TL-PA9020PKIT1 https://www.insight.com/en_US/shop/product/TL-PA9020PKIT1/TP-LINK%20TECHNOLOGY/TL-PA9020P%20KIT/TPLink-Powerline-AV2000--Starter-Kit--powerline-adapter-kit- -wallpluggable/ 101.99 美元 403
TP-Link TL-WR802N - 无线路由器 - 802.11b/g/n - 桌面 TL-WR802N https://www.insight.com/en_US/shop/product/TL-WR802N/TP-LINK%20TECHNOLOGY/TL-WR802N/TPLink-TLWR802N--wireless-router--80211bgn--desktop/ 29.99 美元 710
... ... ... ... ...
于 2022-01-12T07:25:39.773 回答