4

我正在使用 .NET 从 URL 下载数据。对于大多数 URL，它没有问题，但对于一个特定的 URL，当我尝试建立连接时，我遇到了一个非常奇怪的错误。此外，该错误仅在第二次（及后续）尝试发出请求时发生。第一次似乎总是有效。

这是一些演示该问题的示例代码：

string url = "https://health-infobase.canada.ca/src/data/covidLive/covid19.csv";

for (int i = 1; i <= 10; i++)
{
    var req = (HttpWebRequest)WebRequest.Create(url);

    // Just in case, rule these out as being related to the issue.
    req.AllowAutoRedirect = false;
    req.ServerCertificateValidationCallback = (object s, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) => true;

    try
    {
        // This line throws the exception.
        using (req.GetResponse()) { }
    }
    catch (Exception ex) {
        Console.WriteLine(ex.ToString());
        Console.WriteLine($"Failed on attempt {i}.");
        return;
    }
}

备注：

  • 使用指定 URL 以外的任何其他 URL 似乎都有效。甚至同一服务器上的其他 URL（使用相同的证书）也可以正常工作。例如，https://health-infobase.canada.ca/pass。
  • 我将 SChannel 日志记录级别提高到 3（警告和错误），但在 Windows 事件日志中没有看到来自 SChannel 源的任何内容。
  • 该问题发生在 .NET 4.8 (528372) 和 .NET Core 3.1.7
  • 问题发生在两者WebRequest和WebClient
  • 在 .NET Framework 4.8 中，使用时问题似乎消失了WebClient.DownloadData()，但使用时仍然出现WebClient.OpenRead()
  • 在 .NET Framework 4.8 中，问题似乎只发生在下载某些文件的 URL 上（就像我的代码示例中的那个）。但是，在 .NET Core 中，路径在https://health-infobase.canada.ca/src/下的任何 URL 都会发生错误。
  • 如果我使用中间 HTTPS 嗅探器（如 Fiddler），那么问题就会消失。
  • 在 Linux 上的 .NET Core 上运行相同的代码不会出现任何问题。
  • 使用硬连线证书验证回调（ServicePointManager.ServerCertificateValidationCallback总是返回true）没有帮助。

我在.NET Core中运行时的堆栈跟踪如下所示：

System.Net.WebException: The SSL connection could not be established, see inner exception. Authentication failed, see inner exception.
 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
 ---> System.ComponentModel.Win32Exception (0x80090330): The specified data could not be decrypted.
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Security.SslStream.ThrowIfExceptional()
   at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at System.Net.HttpWebRequest.SendRequest()
   at System.Net.HttpWebRequest.GetResponse()
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at UserQuery.Main() in C:\Users\robs\AppData\Local\Temp\LINQPad6\_gifldqtg\xltrxu\LINQPadQuery:line 12

在.NET Framework上，堆栈跟踪似乎没那么有用：

System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.
   at System.Net.HttpWebRequest.GetResponse()
   at UserQuery.Main() in C:\Users\robs\AppData\Local\Temp\LINQPad5\_psduzptv\dcrjhq\LINQPadQuery.cs:line 48

更新：在 github 上作为问题提交：https ://github.com/dotnet/runtime/issues/43682

4

2 回答 2

0

更新

我又挖了一些，并试图强行关闭与服务器的 SSL 连接。可悲的是，没有 API，或者我只是没有找到。所以我们会玩一点反射。


for (int i = 1; i <= 25; i++)
{
    try
    {
        Console.WriteLine(i);

        // use new instances everytime now
        using var handler = new HttpClientHandler();
        using var client = new HttpClient(handler);

        // I found the stream contains a reference to the connection...
        await using var test = await client.GetStreamAsync(url);

        var connectionField = test.GetType()
            .GetField("_connection", BindingFlags.Instance | BindingFlags.NonPublic);
        var connection = connectionField.GetValue(test);

        // ...which contains a reference to the SslStream...
        var sslStreamField = connection.GetType()
            .GetField("_stream", BindingFlags.Instance | BindingFlags.NonPublic);
        var sslStream = sslStreamField.GetValue(connection) as SslStream;


        using var sr = new StreamReader(test);
        var data = await sr.ReadToEndAsync();
        Console.WriteLine(data.Substring(0, 100));

        // ... which I'll shutdown now.
        sslStream.Close();
        sslStream.Dispose();

        await Task.Delay(1000);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Failed on attempt {i} : {ex.Message}.");
    }
}

这工作正常，我们在 Wireshark 中看到了我们的朋友 FIN、ACK。有趣的是，错误仍然以交替模式发生。也许有一些损坏的负载平衡器或正在发生的事情 - 我不知道。

这是输出

1
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
2
Failed on attempt 2 : The SSL connection could not be established, see inner exception..
3
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
4
Failed on attempt 4 : The SSL connection could not be established, see inner exception..
5
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
6
Failed on attempt 6 : The SSL connection could not be established, see inner exception..
7
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
8
Failed on attempt 8 : The SSL connection could not be established, see inner exception..
9
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
10
Failed on attempt 10 : The SSL connection could not be established, see inner exception..
11
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
12
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
13
Failed on attempt 13 : The SSL connection could not be established, see inner exception..
14
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
15
Failed on attempt 15 : The SSL connection could not be established, see inner exception..
16
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
17
Failed on attempt 17 : The SSL connection could not be established, see inner exception..
18
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra
19
Failed on attempt 19 : The SSL connection could not be established, see inner exception..
20
pruid,prname,prnameFR,date,numconf,numprob,numdeaths,numtotal,numtested,numrecover,percentrecover,ra

旧答案（更好的 IMO）

不确定这里的问题是什么，但好吧，这里有一个解决方案。

    static string url = "https://health-infobase.canada.ca/src/data/covidLive/covid19.csv";

    static async Task Fix1()
    {
        var client = new HttpClient();
        for (int i = 1; i <= 25; i++)
        {
            var response = await client.GetAsync(url);
            // .. do something
        }
    }

    static async Task Fix2()
    {
        var handler = new HttpClientHandler();
        for (int i = 1; i <= 25; i++)
        {
            var client = new HttpClient(handler);
            var response = await client.GetAsync(url);
            // .. do something
        }
    }

确实看起来很奇怪。保留客户端或处理程序将使连接保持活动状态并防止另一个密钥交换。

让我们再次修改您的示例并发送一个 UDP 触发数据包，以便我们可以看到 GetResponse 之后会发生什么。

static async Task WhatIsGoingOn()
{
    UdpClient udp = new UdpClient();
    for (int i = 1; i <= 25; i++)
    {
        var req = (HttpWebRequest)WebRequest.CreateHttp(url);
        using var data = req.GetResponse();

        await udp.SendAsync(new byte[] { 1, 2, 3, 4 }, 4, IPEndPoint.Parse("255.255.255.255"));
        GC.Collect();
        GC.WaitForFullGCComplete();
        GC.WaitForPendingFinalizers();
        await Task.Delay(200);
    }
}

在 Wireshark 中，我们将确保在发送触发数据包时传输仍在进行中。启动的另一个 TLS 连接被服务器重置。也许是速率限制或限制与服务器的活动连接？

如果您使用原始循环并使用 sleeps / Task.Delays 进行节流，您将获得 3/4 或更多“成功”连接。

即使块范围丢失并且使用应该完成它的工作或者如果手动调用 Dispose ，连接仍然存在的事实实际上非常奇怪。

于 2020-10-21T00:39:04.973 回答
0

根据https://github.com/dotnet/runtime/issues/43682中的最新信息，这似乎是 Windows 中的操作系统错误。将 Windows 更新到最新版本可以解决此问题。

于 2021-02-17T21:37:07.480 回答