python - ConnectionResetError: [WinError 10054] 现有连接被远程主机强行关闭 - 收到此错误

Question

ConnectionResetError: [WinError 10054] 现有连接被远程主机强行关闭

从 urllib.request import urlopen as uReq 读取以下代码中的网页时出现此错误

from bs4 import BeautifulSoup as soup

myurl = 'https://www.amazon.in/s?k=graphics+card&ref=nb_sb_noss_2'

uClient =uReq(myurl)

Answer 1

传递useragent标头似乎可以解决问题。

尝试这样的事情：

from urllib.request import urlopen as uReq, Request

from bs4 import BeautifulSoup as soup

headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.3"}

my_url = 'https://www.amazon.in/s?k=graphics+card&ref=nb_sb_noss_2'

uClient = uReq(Request(url=my_url, headers=headers))

但是要知道，如果您尝试抓取的数据是动态的，那bs4将无济于事。考虑使用pyppeteerorselenium等​​。