6

我想我已经设法进行了一个可重复显示此问题的测试，至少在我的系统上是这样。这个问题与 HttpClient 被用于错误的端点（不存在的端点，目标已关闭）有关。

问题是已完成任务的数量少于总数，通常约为几个。我不介意请求不起作用，但这只会导致应用程序在等待结果时挂在那里。

我从下面的测试代码中得到以下结果：

经过：237.2009884 秒。批处理数组中的任务：8000 已完成的任务：7993

如果我将 batchsize 设置为 8 而不是 8000，它就完成了。对于 8000，它会卡在 WhenAll 上。

我想知道其他人是否得到相同的结果，如果我做错了什么，以及这似乎是一个错误。

using System;
using System.Diagnostics;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

namespace CustomArrayTesting
{

    /// <summary>
    /// Problem: a large batch of async http requests is done in a loop using HttpClient, and a few of them never complete
    /// </summary>
    class ProgramTestHttpClient
    {
        static readonly int batchSize = 8000; //large batch size brings about the problem

        static readonly Uri Target = new Uri("http://localhost:8080/BadAddress");

        static TimeSpan httpClientTimeout = TimeSpan.FromSeconds(3);  // short Timeout seems to bring about the problem.

        /// <summary>
        /// Sends off a bunch of async httpRequests using a loop, and then waits for the batch of requests to finish.
        /// I installed asp.net web api client libraries Nuget package.
        /// </summary>
        static void Main(String[] args)
        {
            httpClient.Timeout = httpClientTimeout; 

            stopWatch = new Stopwatch();
            stopWatch.Start();


            // this timer updates the screen with the number of completed tasks in the batch (See timerAction method bellow Main)
            TimerCallback _timerAction = timerAction;
            TimerCallback _resetTimer = ResetTimer;
            TimerCallback _timerCallback = _timerAction + _resetTimer;

            timer = new Timer(_timerCallback, null, TimeSpan.FromSeconds(1), Timeout.InfiniteTimeSpan);
            //

            for (int i = 0; i < batchSize; i++)
            {
                Task<HttpResponseMessage> _response = httpClient.PostAsJsonAsync<Object>(Target, new Object());//WatchRequestBody()

                Batch[i] = _response;
            }

            try
            {
                Task.WhenAll(Batch).Wait();
            }
            catch (Exception ex)
            {

            }

            timer.Dispose();
            timerAction(null);
            stopWatch.Stop();


            Console.WriteLine("Done");
            Console.ReadLine();
        }

        static readonly TimeSpan timerRepeat = TimeSpan.FromSeconds(1);

        static readonly HttpClient httpClient = new HttpClient();

        static Stopwatch stopWatch;

        static System.Threading.Timer timer;

        static readonly Task[] Batch = new Task[batchSize];

        static void timerAction(Object state)
        {
            Console.Clear();
            Console.WriteLine("Elapsed: {0} seconds.", stopWatch.Elapsed.TotalSeconds);
            var _tasks = from _task in Batch where _task != null select _task;
            int _tasksCount = _tasks.Count();

            var _completedTasks = from __task in _tasks where __task.IsCompleted select __task;
            int _completedTasksCount = _completedTasks.Count();

            Console.WriteLine("Tasks in batch array: {0}       Completed Tasks : {1} ", _tasksCount, _completedTasksCount);

        }

        static void ResetTimer(Object state)
        {
            timer.Change(timerRepeat, Timeout.InfiniteTimeSpan);
        }
    }
}

有时它只是在以访问冲突未处理异常结束之前崩溃。调用堆栈只是说：

>   mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode = 1225, uint numBytes = 0, System.Threading.NativeOverlapped* pOVERLAP = 0x08b38b98) 
    [Native to Managed Transition]  
    kernel32.dll!@BaseThreadInitThunk@12()  
    ntdll.dll!___RtlUserThreadStart@8()     
    ntdll.dll!__RtlUserThreadStart@8()

大多数时候它不会崩溃，但永远不会完成等待。在任何情况下，每个请求都会抛出以下第一次机会异常：

A first chance exception of type 'System.Net.Sockets.SocketException' occurred in System.dll
A first chance exception of type 'System.Net.WebException' occurred in System.dll
A first chance exception of type 'System.AggregateException' occurred in mscorlib.dll
A first chance exception of type 'System.ObjectDisposedException' occurred in System.dll

我让调试器在对象处理异常上停止，并得到了这个调用堆栈：

>   System.dll!System.Net.Sockets.NetworkStream.UnsafeBeginWrite(byte[] buffer, int offset, int size, System.AsyncCallback callback, object state) + 0x136 bytes    
    System.dll!System.Net.PooledStream.UnsafeBeginWrite(byte[] buffer, int offset, int size, System.AsyncCallback callback, object state) + 0x19 bytes  
    System.dll!System.Net.ConnectStream.WriteHeaders(bool async = true) + 0x105 bytes   
    System.dll!System.Net.HttpWebRequest.EndSubmitRequest() + 0x8a bytes    
    System.dll!System.Net.HttpWebRequest.SetRequestSubmitDone(System.Net.ConnectStream submitStream) + 0x11d bytes  
    System.dll!System.Net.Connection.CompleteConnection(bool async, System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}) + 0x16c bytes  
    System.dll!System.Net.Connection.CompleteConnectionWrapper(object request, object state) + 0x4e bytes   
    System.dll!System.Net.PooledStream.ConnectionCallback(object owningObject, System.Exception e, System.Net.Sockets.Socket socket, System.Net.IPAddress address) + 0xf0 bytes 
    System.dll!System.Net.ServicePoint.ConnectSocketCallback(System.IAsyncResult asyncResult) + 0xe6 bytes  
    System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken) + 0x65 bytes    
    System.dll!System.Net.ContextAwareResult.Complete(System.IntPtr userToken) + 0x92 bytes 
    System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken) + 0xa6 bytes  
    System.dll!System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped) + 0x98 bytes 
    mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) + 0x6e bytes    
    [Native to Managed Transition]

异常消息是：

{"Cannot access a disposed object.\r\nObject name: 'System.Net.Sockets.NetworkStream'."}    System.Exception {System.ObjectDisposedException}

请注意与我很少看到的未处理访问冲突异常的关系。

因此，似乎 HttpClient 在目标关闭时并不健壮。顺便说一句，我在 Windows 7 32 上这样做。

4

3 回答 3

3

我使用反射器查看了 HttpClient 的源代码。对于操作的同步执行部分（当它被启动时），据我所见，似乎没有对返回的任务应用超时。有一些在 HttpWebRequest 对象上调用 Abort() 的超时实现，但它们似乎再次错过了异步函数这一侧的返回任务的任何超时取消或故障。回调方面可能有一些东西，但有时回调可能“丢失”，导致返回的任务永远不会完成。

我发布了一个问题，询问如何为任何任务添加超时，回答者给出了这个非常好的解决方案（这里作为扩展方法）：

public static Task<T> WithTimeout<T>(this Task<T> task, TimeSpan timeout)
{
    var delay = task.ContinueWith(t => t.Result
        , new CancellationTokenSource(timeout).Token);
    return Task.WhenAny(task, delay).Unwrap();
}

因此，像这样调用 HttpClient 应该可以防止任何“任务变坏”永无止境：

Task<HttpResponseMessage> _response = httpClient.PostAsJsonAsync<Object>(Target, new Object()).WithTimeout<HttpResponseMessage>(httpClient.Timeout);

我认为还有几件事使请求丢失的可能性降低： 1. 将超时时间从 3 秒增加到 30 秒，使我发布的程序中的所有任务都完成了这个问题。2. 增加允许的并发连接数，例如 System.Net.ServicePointManager.DefaultConnectionLimit = 100;

于 2013-11-28T23:02:16.983 回答
2

当我从 WCF 搜索类似问题的解决方案时，我遇到了这个问题。这一系列异常与我看到的模式完全相同。最终通过大量调查，我在 HttpClient 使用的 HttpWebRequest 中发现了一个错误。HttpWebRequest 处于错误状态，仅发送 HTTP 标头。然后它等待一个永远不会发送的响应。

我已经用 Microsoft Connect 提出了一张票，可以在这里找到：https ://connect.microsoft.com/VisualStudio/feedback/details/1805955/async-post-httpwebrequest-hangs-when-a-socketexception-occurs-during -setsocket选项

详细信息在票证中，但它需要从 HttpWebRequest 到非本地计算机的异步 POST 调用。我已经在 .Net 4.5 和 4.6 的 Windows 7 上复制了它。引发 SocketException 的失败的 SetSocketOption 调用仅在 Windows 7 上的测试中失败。

对我们来说，UseNagleAlgorithm 设置会导致 SetSocketOption 调用，但我们无法避免它，因为 WCF 关闭了 UseNagleAlgorithm 并且您无法停止它。在 WCF 中，它显示为超时调用。显然这不是很好，因为我们花了 60 年代等待什么。

于 2015-09-21T10:35:24.053 回答
1

您的异常信息在WhenAll任务中丢失。而不是使用它，试试这个：

Task aggregateTask = Task.Factory.ContinueWhenAll(
    Batch,
    TaskExtrasExtensions.PropagateExceptions,
    TaskContinuationOptions.ExecuteSynchronously);

aggregateTask.Wait();

这使用了PropagateExceptionsParallel Extensions Extras 示例代码中的扩展方法来确保批处理操作中的任务的异常信息不会丢失：

/// <summary>Propagates any exceptions that occurred on the specified tasks.</summary>
/// <param name="tasks">The Task instances whose exceptions are to be propagated.</param>
public static void PropagateExceptions(this Task [] tasks)
{
    if (tasks == null) throw new ArgumentNullException("tasks");
    if (tasks.Any(t => t == null)) throw new ArgumentException("tasks");
    if (tasks.Any(t => !t.IsCompleted)) throw new InvalidOperationException("A task has not completed.");
    Task.WaitAll(tasks);
}
于 2013-11-09T20:22:16.870 回答