2

我正在发出一个 Web 请求,将用户重定向到另一个 URL 并获取数据。在浏览器中,它会正确重定向并从重定向的 URL 返回结果。但是,它不适用于控制台应用程序:

HttpWebRequest webrequest = (HttpWebRequest)WebRequest.Create(url1);
webrequest.KeepAlive = true;
webrequest.Method = "GET";
webrequest.ContentType = "text/plain";
webrequest.Timeout = 20000;

WebResponse webresponse = webrequest.GetResponse();
Encoding enc = System.Text.Encoding.GetEncoding("utf-8");
StreamReader loResponseStream = new StreamReader(webresponse.GetResponseStream(), enc);

string result = loResponseStream.ReadToEnd();

我得到的结果是:

<html><body>You are being <a href=\"..........................\">redirected</a>.</body></html>

所以我需要从href锚标签的 URL 中获取数据。

var matches = Regex.Matches(result, @"<a\shref=""(?<url>.*?)"">(?<text>.*?)</a>");
Console.WriteLine(matches[0].Groups["url"].Value);

webrequest = (HttpWebRequest)WebRequest.Create(matches[0].Groups["url"].Value);

webresponse = (HttpWebResponse)webrequest.GetResponse();
enc = System.Text.Encoding.GetEncoding("utf-8");
loResponseStream = new StreamReader(webresponse.GetResponseStream(), enc);

result = loResponseStream.ReadToEnd();

但是我在第二个请求中收到了 403: Forbidden 错误。

随着Fiddler,响应是:

<?xml version="1.0" encoding="UTF-8"?> <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><StringToSignBytes>47 45 54 0a 0a 74 65 78 74 2f 70 6c 61 69 6e 0a 31 33 37 35 38 36 34 31 31 32 0a 2f 61 70 70 74 68 6f 72 69 74 79 2d 73 74 61 74 69 63 2d 72 65 70 6f 72 74 73 2f 36 37 66 62 34 35 39 62 65 63 35 66 63 34 39 37 30 61 32 39 65 64 30 33 61 64 37 30 64 30 31 32 2f 65 6e 2e 6a 73 6f 6e</StringToSignBytes><RequestId>04FB1EFE872CE953</RequestId><HostId>z5KnzkJ2+qzkz1Dad80OPACAvbUbQppLTBRzVp6JiPIIAEWWKi9nckBdXLk5pHzu</HostId><SignatureProvided>kGtsbufDT6eOIDuHnVfDGY20XMA=</SignatureProvided><StringToSign>GET

这里Signature发送到第二个请求与我在href第一个响应的 URL 中收到的相同。

此外,这两个请求Connection: keep-alive在浏览器中都有标头,但在控制台应用程序中,只有第一个请求具有该标头(即使在 之后webrequest.KeepAlive = true;)。

4

1 回答 1

0

尝试添加用户代理:

request.UserAgent = @"Mozilla/5.0 (compatible; Rigor/1.0.0; http://rigor.com)";
于 2021-09-22T13:59:53.883 回答