0

这是对此处提出的问题的扩展： Selenium Python Pass Date parameter using Calendar

使用硒是不可行的，因为我想要过去 5 年的所有日期的数据，导航日历控件会很困难 - 因此我正在尝试使用请求 - 发布方法。下面是不获取数据的代码。但是我无法在这里更改日期。因为更改日期会更改 和 的__VIEWSTATE值 __EVENTVALIDATION。在下面的代码中，复制了 2021 年 12 月 29 日的 __viewstate 和 __eventvalidation。因此，仅显示 2021 年 12 月 29 日的数据。我希望为过去 5 年的所有有效日期运行一个循环，但我无法通过日期作为后参数。但是我可以更改&ctl00%24MainContent%24drpselectinscompany值并将其更改为所需的公司

有没有办法为日期设置这些值？我不确定是否有任何 js 正在这样做。如果是这样，如何使用 Python 来实现

import requests
from requests.structures import CaseInsensitiveDict
from bs4 import BeautifulSoup
import pandas as pd
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

url = "https://www.lifeinscouncil.org/industry%20information/list_of_fund_NAVs.aspx"

headers = CaseInsensitiveDict()
headers["Connection"] = "keep-alive"
headers["Cache-Control"] = "max-age=0"
headers["sec-ch-ua-mobile"] = "?0"
headers["sec-ch-ua-platform"] = "Windows"
headers["Upgrade-Insecure-Requests"] = "1"
headers["Origin"] = "https://www.lifeinscouncil.org"
headers["Content-Type"] = "application/x-www-form-urlencoded"
headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"
headers["Accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
headers["Sec-Fetch-Site"] = "same-origin"
headers["Sec-Fetch-Mode"] = "navigate"
headers["Sec-Fetch-User"] = "?1"
headers["Sec-Fetch-Dest"] = "document"
headers["Referer"] = "https://www.lifeinscouncil.org/industry%20information/list_of_fund_NAVs.aspx"
headers["Accept-Language"] = "en-US,en;q=0.9"
# headers["Cookie"] = "ASP.NET_SessionId=bc24lc0kaaemmnf1w5tntvhr; _ga=GA1.2.2023126567.1641271698"

# data = "__VIEWSTATE=%2FwEPDwUJNDU5MDM3NTQ4D2QWAmYPZBYEAgMPZBYGAgUPD2QWAh4MYXV0b2NvbXBsZXRlBQNvZmZkAgcPD2QWAh8ABQNvZmZkAg0PZBYIAgEPEA8WBh4NRGF0YVRleHRGaWVsZAUMY29tcGFueV9uYW1lHg5EYXRhVmFsdWVGaWVsZAUOY29tcGFueV9yZWZfaWQeC18hRGF0YUJvdW5kZ2QQFRkSLS1TZWxlY3QgSW5zdXJlci0tL0FkaXR5YSBCaXJsYSBTdW4gTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkJEFlZ29uIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCxBR0VBUyBGZWRlcmFsIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCRBdml2YSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQsQmFqYWogQWxsaWFueiBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQpQmhhcnRpIEF4YSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWREQ2FuYXJhIEhTQkMgT3JpZW50YWwgQmFuayBvZiBDb21tZXJjZSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQuRWRlbHdlaXNzIFRva2lvIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCRFeGlkZSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQ0RnV0dXJlIEdlbmVyYWxpIEluZGlhIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCNIREZDIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZC9JQ0lDSSBQcnVkZW50aWFsIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZClJbmRpYUZpcnN0IExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZC1Lb3RhayBNYWhpbmRyYSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQiTWF4IExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCpQTkIgTWV0bGlmZSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQhUHJhbWVyaWNhIExpZmUgSW5zdXJhbmNlIExpbWl0ZWQuLlJlbGlhbmNlIE5pcHBvbiBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQrU2FoYXJhIEluZGlhIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCJTQkkgTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkJlNocmlyYW0gTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkMlN0YXIgVW5pb24gRGFpLUljaGkgTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkJ1RBVEEgQUlBIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCNMaWZlIEluc3VyYW5jZSBDb3Jwb3JhdGlvbiBPZiBJbmRpYRUZATAGQkxTMTExBkFFRzExMQZJREIxMTEGQVZJMTExBkJBSjExMQZCQVgxMTEGQ0hPMTExBkVETDExMQZFWEwxMTEGRkdJMTExBkhERjExMQZJQ1AxMTEGSUZMMTExBktNTzExMQZNWEwxMTEGUEJOMTExBkRIRjExMQZSRUwxMTEGU0FIMTExBlNCSTExMQZTSFIxMTEGU1RSMTExBlRBVDExMQZMSUMxMTEUKwMZZ2dnZ2dnZ2dnZ2dnZ2dnZ2dnZ2dnZ2dnZxYBAgFkAgMPZBYCAgEPZBYCZg9kFgJmDzwrAAoBAA8WBB4HVmlzaWJsZWgeAlNEFgEGAMBG%2FEvO2QhkZAIJDw8WAh8EaGRkAgsPPCsAEQEMFCsAAGQCDQ8PFgIeBFRleHQFBzU0MjgwMDhkZBgCBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WAQUgY3RsMDAkTWFpbkNvbnRlbnQkaW1nYnRuY2FsZW5kZXIFG2N0bDAwJE1haW5Db250ZW50JEdyaWRWaWV3MQ9nZL5eonec14cIkaOIliK6Q1mMNAR%2FU2P%2F%2B6ev%2FYqWD20G\
# &__VIEWSTATEGENERATOR=42AAE71E\
# &__EVENTVALIDATION=%2FwEdACAwNpta9T%2FSrvzXAuzS015l1WCnTNN%2BKNQRDxv9qAhEeUph9FmZgJLYPfmudHQkSHqUM2QXjKqMZLlcCJYOpQfgmXx8lUSq0lByT7%2F%2BPmCayIit0p6J5laSYGRXeVJ%2BZci5ptwMeGZNV6JWrXKJ3w55XRXlQRPkjrxfNAxXvLiY%2FPB%2FLeA4wPkUsYH1Nkq1DrGiaFR%2BmhffK4BGpCW9TO7fgTlrGM9aX38u5yrPE0F0mEPchH7DsatI2Y%2FvDsZXgAuZ4aUhXrLDJpRK%2Ff3L4ROYVbIiFyJxNAKtHqbZ1mzl4iLfxLaRBQ%2BoRsX%2FeBsp0mXQ5bZrXAw2dBu8bbQDF5FuD5y4FlgHeiURhqDskJGqV%2FI%2BZDQOWkVL24K9tz6BYpJcW5mPAbkRiHA2GMxbQE3Qdj62YouF3aa0UjC38b58UjdJI0VtSG3jcUk1ZZRHSNKvmq6rubYiTmISD8H87RX7sjklmTDbRuRba%2FWbhtomy9PxsXOQXE5%2FlbPFElCY9pXzXtWOtaAH6T9GE%2BpGSVYHWk6zbIiGnJ%2Bmuh0wRHecp%2FxE6HYEfhrGudMxiXQRBvlY%2BQR2qch2GWeJVenQqG%2FPibgABw%2F5bYtC7JPASNV0SLwOWq70UnY8OnmZdgHxm7YHd9QB0m473mX3OFskbsfD8rEL7grjIt6k0Md01JR%2FQNtWi%2FmqnWZWr6d%2BiWhWW%2B40DQ09kC0yB7qcRlAI8%2Fm%2B\
# &ctl00%24txtusername=\
# &ctl00%24txtpwd=\
# &ctl00%24MainContent%24drpselectinscompany=*Insurer*\
# &ctl00%24MainContent%24txtdateselect=*DATE*\
# &ctl00%24MainContent%24btngetdetails=Get+Data"

data = "__VIEWSTATE=%2FwEPDwUJNDU5MDM3NTQ4D2QWAmYPZBYEAgMPZBYGAgUPD2QWAh4MYXV0b2NvbXBsZXRlBQNvZmZkAgcPD2QWAh8ABQNvZmZkAg0PZBYIAgEPEA8WBh4NRGF0YVRleHRGaWVsZAUMY29tcGFueV9uYW1lHg5EYXRhVmFsdWVGaWVsZAUOY29tcGFueV9yZWZfaWQeC18hRGF0YUJvdW5kZ2QQFRkSLS1TZWxlY3QgSW5zdXJlci0tL0FkaXR5YSBCaXJsYSBTdW4gTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkJEFlZ29uIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCxBR0VBUyBGZWRlcmFsIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCRBdml2YSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQsQmFqYWogQWxsaWFueiBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQpQmhhcnRpIEF4YSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWREQ2FuYXJhIEhTQkMgT3JpZW50YWwgQmFuayBvZiBDb21tZXJjZSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQuRWRlbHdlaXNzIFRva2lvIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCRFeGlkZSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQ0RnV0dXJlIEdlbmVyYWxpIEluZGlhIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCNIREZDIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZC9JQ0lDSSBQcnVkZW50aWFsIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZClJbmRpYUZpcnN0IExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZC1Lb3RhayBNYWhpbmRyYSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQiTWF4IExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCpQTkIgTWV0bGlmZSBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQhUHJhbWVyaWNhIExpZmUgSW5zdXJhbmNlIExpbWl0ZWQuLlJlbGlhbmNlIE5pcHBvbiBMaWZlIEluc3VyYW5jZSBDb21wYW55IExpbWl0ZWQrU2FoYXJhIEluZGlhIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCJTQkkgTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkJlNocmlyYW0gTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkMlN0YXIgVW5pb24gRGFpLUljaGkgTGlmZSBJbnN1cmFuY2UgQ29tcGFueSBMaW1pdGVkJ1RBVEEgQUlBIExpZmUgSW5zdXJhbmNlIENvbXBhbnkgTGltaXRlZCNMaWZlIEluc3VyYW5jZSBDb3Jwb3JhdGlvbiBPZiBJbmRpYRUZATAGQkxTMTExBkFFRzExMQZJREIxMTEGQVZJMTExBkJBSjExMQZCQVgxMTEGQ0hPMTExBkVETDExMQZFWEwxMTEGRkdJMTExBkhERjExMQZJQ1AxMTEGSUZMMTExBktNTzExMQZNWEwxMTEGUEJOMTExBkRIRjExMQZSRUwxMTEGU0FIMTExBlNCSTExMQZTSFIxMTEGU1RSMTExBlRBVDExMQZMSUMxMTEUKwMZZ2dnZ2dnZ2dnZ2dnZ2dnZ2dnZ2dnZ2dnZxYBAgRkAgMPZBYCAgEPZBYCZg9kFgJmDzwrAAoBAA8WBB4HVmlzaWJsZWgeAlNEFgEGAAA2KF7K2QhkZAIJDw8WAh8EaGRkAgsPPCsAEQEMFCsAAGQCDQ8PFgIeBFRleHQFBzU0MzM3MzBkZBgCBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WAQUgY3RsMDAkTWFpbkNvbnRlbnQkaW1nYnRuY2FsZW5kZXIFG2N0bDAwJE1haW5Db250ZW50JEdyaWRWaWV3MQ9nZFEaucYeLzMSOdVUP6KUFN35Bp81FRdiApv6vjNaF1Zj\
&__VIEWSTATEGENERATOR=\
&__EVENTVALIDATION=%2FwEdACDmqAABWGOpTJ8hvZs8guEA1WCnTNN%2BKNQRDxv9qAhEeUph9FmZgJLYPfmudHQkSHqUM2QXjKqMZLlcCJYOpQfgmXx8lUSq0lByT7%2F%2BPmCayIit0p6J5laSYGRXeVJ%2BZci5ptwMeGZNV6JWrXKJ3w55XRXlQRPkjrxfNAxXvLiY%2FPB%2FLeA4wPkUsYH1Nkq1DrGiaFR%2BmhffK4BGpCW9TO7fgTlrGM9aX38u5yrPE0F0mEPchH7DsatI2Y%2FvDsZXgAuZ4aUhXrLDJpRK%2Ff3L4ROYVbIiFyJxNAKtHqbZ1mzl4iLfxLaRBQ%2BoRsX%2FeBsp0mXQ5bZrXAw2dBu8bbQDF5FuD5y4FlgHeiURhqDskJGqV%2FI%2BZDQOWkVL24K9tz6BYpJcW5mPAbkRiHA2GMxbQE3Qdj62YouF3aa0UjC38b58UjdJI0VtSG3jcUk1ZZRHSNKvmq6rubYiTmISD8H87RX7sjklmTDbRuRba%2FWbhtomy9PxsXOQXE5%2FlbPFElCY9pXzXtWOtaAH6T9GE%2BpGSVYHWk6zbIiGnJ%2Bmuh0wRHecp%2FxE6HYEfhrGudMxiXQRBvlY%2BQR2qch2GWeJVenQqG%2FPibgABw%2F5bYtC7JPASNV0SLwOWq70UnY8OnmZdgHxm7YHd9QB0m473mX3OFskbsfD8rEL7grjIt6k0Md01JR%2FQBbHdHfLX41jNJsWaST37PPucd7HiYowfstiUHMRIM4y\
&ctl00%24txtusername=\
&ctl00%24txtpwd=\
&ctl00%24MainContent%24drpselectinscompany=*Insurer*\
&ctl00%24MainContent%24txtdateselect=*DATE*\
&ctl00%24MainContent%24btngetdetails=Get+Data"

insurer='BLS111'
date='31-Dec-21'
data=data.replace('*Insurer*',insurer)
data=data.replace('*DATE*',date)
print(data)
resp = requests.post(url, headers=headers, data=data)
soup=BeautifulSoup(resp.text,'html.parser')
# print(soup.prettify())
table=soup.find_all('table',{'id':'MainContent_GridView1'})
# print(table)
df=pd.read_html(str(table))[0]
print(df)

我也尝试过使用邮递员来帮助我生成代码，但邮递员也无法在输出中获取数据

我在这里想念什么？

感谢您提前提供帮助。问候， 基兰耆那教

4

0 回答 0