python - 填写文本类型的输入并使用python按提交

Question

我有这个html：

<input type="text" class="txtSearch">
<input type="submit" value="Search" class="sbtSearch">

我需要的是在文本字段中写入，然后单击使用 python 提交。输入标签不在Form内。我怎么能那样做？

Answer 1

您不必实际填充字段并“单击”提交。您可以模拟提交并获得所需的结果。

在 Firefox 中与 firebug 一起使用BeautifulSoup和 urllib。使用 firebug 观察网络流量，并从提交正在执行的 HTTP POST 中获取 post 参数。创建一个 dict 并对其进行 url 编码。将它与您的 url 请求一起传递。

例如：

from BeautifulSoup import BeautifulSoup
import urllib

post_params = {
    param1 : val1,
    param2 : val2,
    param3 : val3
        }
post_args = urllib.urlencode(post_params)

url = 'http://www.website.com/'
fp = urllib.urlopen(url, post_args)
soup = BeautifulSoup(fp)

该参数vals将根据您尝试提交的内容而更改。在您的代码中进行适当的调整。

Answer 2

Here's a selenium solution if you actually need to populate the fields. You would typically only need this for testing purposes, though.

from selenium import webdriver

webpage = r"https://www.yourwebsite.com/" # edit me
searchterm = "Hurricane Sandy" # edit me

driver = webdriver.Chrome()
driver.get(webpage)

sbox = driver.find_element_by_class_name("txtSearch")
sbox.send_keys(searchterm)

submit = driver.find_element_by_class_name("sbtSearch")
submit.click()

Answer 3

UPDATED 2019 answer. This code also takes care of the HTTP 403 Forbidden errors.

import urllib.request as urlRequest
import urllib.parse as urlParse

url = "https://yoururl.com"
values = {"name": "value"}

# pretend to be a chrome 47 browser on a windows 10 machine
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"}

# encode values for the url
params = urlParse.urlencode(values).encode("utf-8")

# create the url
targetUrl = urlRequest.Request(url=url, data=params, headers=headers)

# open the url
x  = urlRequest.urlopen(targetUrl)

# read the response
respone = x.read()
print(respone)