0

所以我尝试使用 Newspaper3k 来抓取一些网站的内容。在库中，该函数Article()只需要一个 url。这是否可以迭代一个充满 url 的数据框来自动抓取它？我的 df 是这样的

df = ['https://www.liputan6.com/bisnis/read/4661489/erick-thohir-apresiasi-transformasi-digital-pos-indonesia','https://ekonomi.bisnis.com/read/20210918/98/1443952/pos-indonesia-gandeng-nujek-perluas-segmen-pengiriman','https://www.republika.co.id/berita/qzkxdm380/perkuat-layanan-pt-pos-indonesia-gandeng-kurir-wanita']

我尝试了一些这样的可能答案

for x in df.iterrows():
    print(x)
a = Article(x,language='id')
b = a.download()
c = a.parse()

但它得到一个错误

AttributeError: 'tuple' object has no attribute 'decode'

我也试试


a = Article(url=x in df.iterrows(),language='id')
b = a.download()
c = a.parse()
author = a.authors
date = a.publish_date
text = a.text

combine = {'author':author,'date':date,'text':text}
data = pd.DataFrame(data=combine)

但出现错误

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

如果得到帮助，我会尝试更多代码，我真的很感激。谢谢

4

1 回答 1

1

df不是数据框，而是列表。只需遍历列表。

from newspaper import Article
import pandas as pd

urls = ['https://www.liputan6.com/bisnis/read/4661489/erick-thohir-apresiasi-transformasi-digital-pos-indonesia','https://ekonomi.bisnis.com/read/20210918/98/1443952/pos-indonesia-gandeng-nujek-perluas-segmen-pengiriman','https://www.republika.co.id/berita/qzkxdm380/perkuat-layanan-pt-pos-indonesia-gandeng-kurir-wanita']

rows = []
for url in urls:
    try:
        a = Article(url,language='id')
        a.download()
        a.parse()
         
        author = a.authors
        date = a.publish_date
        text = a.text
        
        print(author, date, text)
        row = {'url':url,
               'author':author,
               'data':date,
               'text':text}
        
        rows.append(row)
    except Exception as e:
        print(e)
        row = {'url':url,
        'author':'N/A',
        'data':'N/A',
        'text':'N/A'}
        
        rows.append(row)
        
df = pd.DataFrame(rows)
于 2021-10-08T11:15:40.323 回答