0

我对异步上传 blob 的问题感到困惑，希望在这里找到答案。

请先看看我的代码片段，

public void UploadMultipleBlobs(List<string> filelocations, string containerName, AsyncCallback callback = null, string path = null)
    {
                    try
        {

            Parallel.ForEach(filelocations, fileLocation =>
                {

                    //File to Stream
                    MemoryStream str = new MemoryStream();
                    byte[] file = File.ReadAllBytes(fileLocation);
                    str.Write(file, 0, file.Length);
                    str.Seek(0, SeekOrigin.Begin);

                    //Operations
                    if (callback == null)
                        callback = new AsyncCallback(OnUploadCompleted);
                    BlobRequestOptions blobRequestOptions = new BlobRequestOptions();
                    blobRequestOptions.Timeout = new TimeSpan(1, 0, 0);
                    blobRequestOptions.RetryPolicy = retry;

                    CloudBlob currentBlob = container.GetBlobReference(blobName);
                    var result = currentBlob.BeginUploadFromStream(str, blobRequestOptions, callback, new Object[] { currentBlob, str });

                    currentBlob.EndUploadFromStream(result);
                });

        }
        catch
        {
            throw;
        }

    }

private void OnUploadCompleted(IAsyncResult result)
    {
        try
        {
            // Get array passed to callback
            Object[] states = (Object[])result.AsyncState;

            var blob = (CloudBlob)states[0];
            var stream = (MemoryStream)states[1];

            // End the operation
            //blob.EndUploadFromStream(result);

            // Close the stream
            stream.Close();
        }
        catch
        {
            throw;
        }
    }

我需要将 mutil 文件上传到 Azure blob，文件数可能为 10-50,000 个，每个文件约为 10KB-50KB。代码片段目前对我来说很好。但是，如果我在回调中调用 EndUploadFromStream，它在上传超过 2,000 个文件时总是会抛出异常。我的意思是，如果我在上传方法中删除 EndUploadFromStream 并在回调（OnUploadCompleted 方法）中调用 EndUploadFromStream，则会发生异常。异常信息如下：

无法从传输连接读取数据：连接已关闭。，StackTrace：在 Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.get_Result()

在 Microsoft.WindowsAzure.StorageClient.CloudBlob.EndUploadFromStream（IAsyncResult asyncResult）

我不知道为什么会这样......希望得到你们的答案。

谢谢。

4

1 回答 1

0

The Begin/End code looks OK. I notice that you're not doing anything to wait for the asynchronous operations to complete, so the problem may be related to that. eg, if you're running this from a console application then the application may exit before all the uploads have completed and then give you those errors. That would not be a problem if the EndUploadFromStream() call is inside the Parallel.ForEach() because it will cause the Parallel.ForEach() call to block until all the uploads have completed.

So try adding code to wait for all the uploads to complete and see if that fixes it. A simple way would be a counter that is initialized to the total number of uploads, and then decremented (using Interlocked.Decremement() for thread safety) inside of the callback. Another options would be use Task.FromAsync() to get an array of Task objects, then using Task.WaitAll() to wait for them to complete.

As an aside, using both Parallel.ForEach() and Begin/End methods at the same time is usually redundant - Begin/End is asynchronous already so there's usually no point using multiple threads to invoke it. Since you have such a big list of items it might make some difference in this case, but probably not much. You're probably better off using a simple foreach loop instead of Parallel.ForEach() unless you've actually measured a significant difference.

于 2012-09-28T16:33:08.223 回答