1

我一直在尝试从新闻网站上抓取新闻标题。为此，我遇到了两个python 库，即报纸和beautifulsoup4。使用漂亮的汤库，我已经能够从一个特定的新闻网站获得所有指向新闻文章的链接。从下面的代码中，我已经能够从单个链接中提取新闻文章的标题。

from newspaper import Article
url= "https://www.ndtv.com/india-news/tamil-nadu-government-reverses-decision-to-reopen-schools-from-november-16-for-classes-9-12-news-agency-pti-2324199"
article=Article(url)
article.download()
article.parse()
print(article.title)

我想结合两个库的代码，即报纸和beautifulsoup4，这样我作为beautifulsoup库的输出获得的所有链接都应该放在报纸库的url命令中，我得到所有标题链接。下面是 beautfulsoup 的代码，我可以从中提取所有新闻文章的链接。

from bs4 import BeautifulSoup
from bs4.dammit import EncodingDetector
import requests

parser = 'html.parser'  # or 'lxml' (preferred) or 'html5lib', if installed
resp = requests.get("https://www.ndtv.com/coronavirus?pfrom=home-mainnavgation")
http_encoding = resp.encoding if 'charset' in resp.headers.get('content-type', '').lower() else None
html_encoding = EncodingDetector.find_declared_encoding(resp.content, is_html=True)
encoding = html_encoding or http_encoding
soup = BeautifulSoup(resp.content, parser, from_encoding=encoding)

for link in soup.find_all('a', href=True):
    print(link['href'])
4

1 回答 1

1

你的意思是这样的吗？

links = []
for link in soup.find_all('a', href=True):
    links.append(link['href'])

for link in links:
    article=Article(link)
    article.download()
    article.parse()
    print(article.title)
于 2020-11-20T11:20:33.927 回答