python-3.x - 用于抓取文章的报纸 api

Question

我已经使用 python 中的报纸 3k api 来抓取文章。我无法抓取印度时报的文章，从响应其余文章中获取发布日期为空，正在提供适当的文章。

article = Article(url)
article.download()
article.parse()
result=vars(article)
print(result['publish_date']) 

Answer 1

当前版本的Newspaper无法从 Times of India HTML 代码中提取“发布日期”，因为该日期在脚本标签内。您可以使用requests和BeautifulSoup提取此日期。后者嵌入在Newspaper中。我还注意到关键字位于元标记中，因此报纸无法提取这些关键字。我也添加了一些代码来提取关键字。希望下面的代码可以帮助您查询印度时报上的文章。请让我知道，如果你有任何问题。

import requests
import re as regex
from newspaper import Article
from newspaper.utils import BeautifulSoup

base_url = 'https://timesofindia.indiatimes.com/business/india-business/govt-working-to-reduce-e-vehicle-tax-niti-aayog-ceo/articleshow/78210495.cms'

raw_html = requests.get(base_url)
soup = BeautifulSoup(raw_html.text, 'html.parser')

# parse date published
data = soup.findAll('script')[1]
find_date = regex.search(r'datePublished.{3}\d{4}-\d{2}-\d{2}', data.string)
date_published = find_date.group().split('"')[2]

# parse other elements using Newspaper
article = Article('')
article.download(raw_html.content)
article.parse()
article_tags = article.tags
article_content = article.text
article_title = article.title

# parse keywords
article_meta_data = article.meta_data
article_keywords = sorted({value for (key, value) in article_meta_data.items() if key == 'keywords'})