0

我的网络抓取工具遇到问题。

我在 python 中做了一个函数来使用 Elsevier 开发门户上提供的 SCOPUS 搜索 API 获取文章详细信息。但是，在我运行该程序后，它给了我与 Elsevier 门户相比不同的结果集。

Elseveir 门户 - https://www.elsevier.com/ SCOPUS API - https://api.elsevier.com/content/search/scopus?query=all(gene)&apiKey=7f59af901d2d86f78a 1fd60c1bf9426a

代码：

url = 'https://api.elsevier.com/content/search/scopus?query=' + 查询 + '&apiKey=' + scp_api

# response object
response = requests.get(url, headers=headers, timeout=30)
soup = BeautifulSoup(response.content, 'lxml')

# convert resonse into josn
obj = json.loads(soup.text)

######## Find required attributes in the response object
for item in obj['search-results']['entry']:
    try:
        if "prism:Issn" and "prism:issn" not in obj:
            issn = item['prism:eIssn']
        else:
            issn = item['prism:issn']

        resp_obj = {"entities": {"Search Engine": "Elsevier SCOPUS Search Engine",
                                 "items": [
                                     {"DOI": item['prism:doi'],
                                      "Title": item['dc:title'],
                                      "URLs": item['prism:url'],
                                      "Authors": item['dc:creator'],
                                      "Publication Name": item['prism:publicationName'],
                                      "ISSN": issn,
                                      "Cited count": item['citedby-count'],
                                      "Affiliation": item['affiliation'][0]['affilname'],
                                      "Type": item['subtypeDescription'],
                                      "Published date": item['prism:coverDate'],
                                      "Abstract": item['prism:publicationName']
                                      }
                                 ]}}
        # append dict object data
        data.append(resp_obj)
    except Exception as e:  # raise e
        pass
        # print('error scopus:', e)

任何理由或帮助将不胜感激。

谢谢。

4

0 回答 0