1

发生这种情况syncor cp,将大约 1000 个文件从我们的本地服务器复制到 S3 存储桶。源是从 Windows 8 机器到在 Hyper-V 中运行的 Windows Server 2008 R2 VM 的映射网络驱动器。

这是命令:

aws s3 cp X:\folder s3://my-bucket/folder --recursive

它失败的文件似乎没有任何模式。有些更大(最大 30MB),有些只有几 kb。这是错误消息:

HTTPSConnectionPool(host='XXX.s3.amazonaws.com', port=443): Max retries exceeded with url: ... (Caused by <class 'socket.error'>: [Errno 10054] An existing connection was forcibly closed by the remote host)
4

2 回答 2

1

I had the same problem. Notebook with Ubuntu, command aws s3 cp or aws s3 sync were failing as described.

My file was 104285 bytes large, so nothing "too big".

Copying the file over BucketExplorer was working without problems.

On botocore are reported issues, which show similar problems for large files, but no report for small ones.

Then I updated boto library from 2.9.5 to the latest 2.23.0, but the problem has persisted.

Later I was testing my problem file, making it smaller and smaller, until it was published. Finally I have found, the problem is gone even for the original file.

It is hard to say, how and if the problem is resolved.

One can think of a temporary problem on AWS side, but as I was able to publish the file over BucketExplorer on the first try, I guess it was not the case.

Checking AWS Health Dashboard: http://status.aws.amazon.com/ I did not find any problems reported, but you may never rely on finding there all small issues.

Anyway, I would propose updating boto to the latest version as it seems the most likely cause:

 $ pip install --upgrade boto

My AWS CLI was alrady updated before I run into these problems and has version 0.31

My final versions are:

$ pip freeze|grep boto
boto==2.23.0
boto-utils==0.3
botocore==0.31.0
于 2014-01-24T23:11:26.253 回答
-1

问题是映射的网络驱动器。从本地磁盘上的文件夹而不是网络位置复制时,所有文件都成功上传。

于 2013-12-16T23:37:48.480 回答