1

我有一个示例应用程序，它使用适用于 .NET 高级 API 的 AWS 开发工具包将大小为 26 MB 的文件上传到 Amazon Glacier。该代码在没有线程的情况下工作正常，但使用线程池它在下面的行失败

         client.UploadMultipartPart(uploadMPUrequest);

说错误消息：请求被中止：请求被取消。

堆栈跟踪：在 Amazon.Runtime.AmazonWebServiceClient.InvokeConfiguredRequest(AsyncResult asyncResult) 在 Amazon.Runtime.Amazon.Runtime.Amazon.Runtime.Amazon.Runtime.AmazonWebServiceClient.getRequestStreamCallback(IAsyncResult result) 在 Amazon.Runtime.AmazonWebServiceClient.handleHttpWebErrorResponse(AsyncResult asyncResult, WebException we) 在 Amazon.Runtime.AmazonWebServiceClient.InvokeHelper (AsyncResult asyncResult) 在 Amazon.Runtime.AmazonWebServiceClient.Invoke(AsyncResult asyncResult) 在 Amazon.Glacier.AmazonGlacierClient.invokeUploadMultipartPart(UploadMultipartPartRequest uploadMultipartPartRequest, AsyncCallback 回调, 对象状态, 布尔同步) 在 Amazon.Glacier.AmazonGlacierClient.UploadMultipartPart(UploadMultipartPartRequest uploadMultipartPartRequest)

注意：我正在多部分上传数据

请在以下链接中找到我的示例代码： www.page-monitor.com/Downloads/ArchiveUploadMPU.cs

是否有任何并行上传档案的示例代码？

谢谢和问候， 哈西娜

4

2 回答 2

0

这是适用于线程的示例代码。ChunkDetails 是一个自定义库，用于传递 accessID、bucketname、offset 详细信息等。我也在使用 ThrottledStream。

       internal bool UploadUsingHighLevelAPI(String FilePath, ChunkDetails ObjMetaData,
                                            S3Operations.UploadType uploadType,
       Stream inputStream)
       {
       String METHOD_NAME = "UploadUsingHighLevelAPI";
        String keyName;
        String existingBucketName;
        TransferUtilityUploadRequest fileTransferUtilityRequest = null;
        int RetryTimes = 3;
        ThrottledStream throttleStreamObj = null;

        long bps = ThrottledStream.Infinite;
        try
        {


            keyName = ObjMetaData.KeyName;
            existingBucketName = ObjMetaData.BucketName;

            TransferUtility fileTransferUtility = new
                    TransferUtility(ObjMetaData.AccessKeyID,        ObjMetaData.SecretAccessKey);

            FileInfo fin = new FileInfo(FilePath);

            //streamObj = new FileStream(FilePath, FileMode.Open);

            bps = (long)(1024 * ObjMetaData.MaxAvailSpeed * ((double)ObjMetaData.Bandwidth / 100.0));


            throttleStreamObj = new ThrottledStream(ObjMetaData.FileStream, bps);


            System.Collections.Specialized.NameValueCollection metaInfo = new System.Collections.Specialized.NameValueCollection();
            if (ObjMetaData.MetaInfo != null)
            {
                foreach (DictionaryEntry kvp in ObjMetaData.MetaInfo)
                {
                    metaInfo.Add(kvp.Key.ToString(), kvp.Value.ToString());
                }
            }


            long OffDiff = ObjMetaData.EndOffset - ObjMetaData.StartOffset;
            long partSize;
            if (fin.Length >= OffDiff)
            {
                partSize = OffDiff;
            }
            else
                partSize = fin.Length;



            if (uploadType == UploadType.File)
            {
                //fileTransferUtility.Upload(FilePath, existingBucketName, keyName);


                fileTransferUtilityRequest =
                new TransferUtilityUploadRequest()
                .WithBucketName(existingBucketName)
                //.WithFilePath(FilePath)
                .WithStorageClass(S3StorageClass.ReducedRedundancy)
                .WithMetadata(metaInfo)
                .WithPartSize(partSize)
                .WithKey(keyName)
                .WithCannedACL(S3CannedACL.PublicRead)
                .WithTimeout(Int32.MaxValue - 1)
                .WithInputStream(throttleStreamObj) as TransferUtilityUploadRequest;

            }
            else if (uploadType == UploadType.Stream)
            {
                fileTransferUtilityRequest =
               new TransferUtilityUploadRequest()
               .WithBucketName(existingBucketName)
               .WithStorageClass(S3StorageClass.ReducedRedundancy)
               .WithMetadata(metaInfo)
               .WithPartSize(partSize)
               .WithKey(keyName)
               .WithCannedACL(S3CannedACL.PublicRead)
               .WithTimeout(Int32.MaxValue - 1)
               .WithInputStream(throttleStreamObj) as TransferUtilityUploadRequest
               ;
            }



            for (int index = 1; index <= RetryTimes; index++)
            {
                try
                {

                    // Upload part and add response to our list.
                    fileTransferUtility.Upload(fileTransferUtilityRequest);
                    Console.WriteLine(" ====== Upload Done =========");
                    if (eventChunkUploaded != null)
                        eventChunkUploaded(ObjMetaData);
                    break;

                }
                catch (Exception ex)
                {
                    if (index == RetryTimes)
                    {
                        m_objLogFile.LogError(CLASS_NAME, METHOD_NAME + " - Attempt " +
                            index + Environment.NewLine + FilePath, ex);

                        if (eventChunkUploadError != null)
                            eventChunkUploadError(ObjMetaData, ex.Message);

                    }
                }
            }
        }
        catch (Exception ex)
        {
            m_objLogFile.LogError(CLASS_NAME, METHOD_NAME, ex);
            return false;
        }
        finally
        {

            if (throttleStreamObj != null)
            {
                //inputStream1.Close();
                throttleStreamObj = null;
            }
        }

        return true;
   }

如果您遇到任何问题，请告诉我。

于 2013-10-03T05:29:52.357 回答
0

我相信代码中存在竞争条件。我正在研究相同的功能。我很乐意与您分享代码。如果您修复了您发布的代码，我将不胜感激它的链接。最好的问候，布鲁斯

于 2013-09-30T15:47:06.177 回答