amazon-web-services - AWS CLI 从 AWS CLI 获取私有存储桶的下载 S3 URL

Question

我可以使用以下命令将文件成功上传到私有S3 存储桶：

aws s3 cp "myfile.txt" "s3://myfolder/myfile.txt" --region=us-east-1 --output=json

我想发出一个 AWS CLI 命令来为我返回 myfile.txt 的临时 URL 下载，有人知道怎么做吗？

我用谷歌搜索，看起来我必须做一些签名才能获得临时 URL，例如：http ://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html

Answer 1

aws cli 现在支持presign命令。你可以跑

$ aws s3 presign s3://test-bucket/test-file.txt
https://test-bucket/test-file.txt?Expires=1499152189&Signature=some-sha

这将生成一个 URL，您可以与任何人共享它以在 3600 秒内下载该文件。

您可以更改时间段--expires-in

$ aws s3 presign s3://test-bucket/test-file.txt --expires-in 600 

生成的 url 将在 10 分钟后过期。

您可以在 aws cli docs 中阅读有关presign 的更多信息。

Answer 2

您可以使用以下 URL 格式：

https://<bucket-name>.s3.amazonaws.com/<object or key name>

或旧式：

https://s3.amazonaws.com/<bucket-name>/<object or key name>

要使其可访问，您需要允许公开访问您的对象或附加适当的存储桶策略。

例如，以下存储桶策略显示对存储桶zzzyyy对象“yyyeee”的公共访问

$ aws s3 get-object-acl --bucket zzzyyy --key yyyeee
{
    "Owner": {
        "DisplayName": "owner",
        "ID": "Some hash of owner"
    },
    "Grants": [
        {
            "Grantee": {
                "DisplayName": "owner",
                "ID": "Some hash of owner"
            },
            "Permission": "READ"
        },
        {
            "Grantee": {
                "DisplayName": "owner",
                "ID": "Some hash of owner"
            },
            "Permission": "WRITE"
        },
        {
            "Grantee": {
                "DisplayName": "owner",
                "ID": "Some hash of owner"
            },
        "Permission": "READ_ACP"
        },
        {
            "Grantee": {
                "DisplayName": "owner",
                "ID": "Some hash of owner"
            },
            "Permission": "WRITE_ACP"
        },
        {
            "Grantee": {
                "URI": "http://acs.amazonaws.com/groups/global/AllUsers"
            },
            "Permission": "READ"
        },
        {
            "Grantee": {
                "URI": "http://acs.amazonaws.com/groups/global/AllUsers"
            },
            "Permission": "READ_ACP"
        }
    ]
}

您可以在此处查看存储桶策略示例：

http://docs.aws.amazon.com/AmazonS3/latest/dev/AccessPolicyLanguage_UseCases_s3_a.html

您还可以使用 S3 控制台，如下所示：

Answer 3

我四处搜索并决定编写这个脚本来帮助我为 S3 生成 singed-url。

https://github.com/gdbtek/aws-tools

Answer 4

命令错误；代替：

$ aws s3 get-object-acl --bucket zzzyyy --key yyyeee

它应该是：

$ aws s3api get-object-acl --bucket zzzyyy --key yyyeee

Answer 5

或者，您可以查看https://github.com/minio/mc并向我们提供反馈。我们一直希望有一个 CLI Dropbox 界面，当然不是 GUI，而是可编写脚本并实现自动化。所以我们最终实现了一个方便的命令mc来解决两个用例。

• 用于共享 URL 以下载文件（Presigned Get）
• 用于共享curl表单以上传文件（Presigned POST Policy）

以下示例解决了这两个用例。

$ mc share
NAME:
  mc share - Download and upload documents.

USAGE:
  mc share command [arguments...]

COMMANDS:
  download  Generate URL to download documents.
  upload    Generate ‘curl’ command to upload files.
  list      List the shared URLs

这些命令可以在常规意义上使用，也可以在脚本中使用。出于脚本目的，mc提供--json所有命令的输出。

示例mc share download

$ mc share download https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3
https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJVA5BMMU2RHO6IOQ%2F20151109%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20151109T060251Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=3c69928bee38b7b465547f9fa97146ff606480bbead8042c0351840c578a2ceb

$ mc --json share download https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3 | jq .

{
  "expiry": {
    "days": 7
  },
  "downloadUrl": "https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJVA5BMMU2RHO6IOQ%2F20151109%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20151109T060618Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=48f7a521396ac8e36443cc86ce16c2619f9fa81371aad9a309375c933ea48d99",
  "keyName": "https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3"
}

示例mc share upload

$ mc share upload --expire=2h --content-type=image/png https://s3.amazonaws.com/backup/2007-Mar-2/... 
curl https://s3.amazonaws.com/backup -F x-amz-credential=AKIAJVA5BMMU2RHO6IOQ/20151109/us-east-1/s3/aws4_request -F x-amz-date=20151109T061202Z -F x-amz-signature=61c234dd17fc9e5888417ac4546b37d61755e551674a9617f33f5232ec53c8b6 -F Content-Type=image/png -F bucket=backup -F policy=eyJleHBpcmF0aW9uIjoiMjAxNS0xMS0wOVQwODoxMjowMi40MDZaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIkQ29udGVudC1UeXBlIiwiaW1hZ2UvcG5nIl0sWyJlcSIsIiRidWNrZXQiLCJiYWNrdXAiXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsIjIwMDctTWFyLTIvIl0sWyJlcSIsIiR4LWFtei1kYXRlIiwiMjAxNTExMDlUMDYxMjAyWiJdLFsiZXEiLCIkeC1hbXotYWxnb3JpdGhtIiwiQVdTNC1ITUFDLVNIQTI1NiJdLFsiZXEiLCIkeC1hbXotY3JlZGVudGlhbCIsIkFLSUFKVkE1Qk1NVTJSSE82SU9RLzIwMTUxMTA5L3VzLWVhc3QtMS9zMy9hd3M0X3JlcXVlc3QiXV19 -F x-amz-algorithm=AWS4-HMAC-SHA256 -F key=2007-Mar-2/<FILE> -F file=@<FILE>

$ mc --json share upload --expire=2h --content-type=image/png https://s3.amazonaws.com/backup/2007-Mar-2/... | jq .
{
  "expiry": {
    "hours": 2
  },
  "uploadCommand": "curl https://s3.amazonaws.com/backup -F x-amz-signature=4b349a15e76ab403424ec6b1e289336f67c02c427ccc98522d6f654eee5b7de6 -F Content-Type=image/png -F bucket=backup -F policy=eyJleHBpcmF0aW9uIjoiMjAxNS0xMS0wOVQwODoxMzoyMS4zODFaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIkQ29udGVudC1UeXBlIiwiaW1hZ2UvcG5nIl0sWyJlcSIsIiRidWNrZXQiLCJiYWNrdXAiXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsIjIwMDctTWFyLTIvIl0sWyJlcSIsIiR4LWFtei1kYXRlIiwiMjAxNTExMDlUMDYxMzIxWiJdLFsiZXEiLCIkeC1hbXotYWxnb3JpdGhtIiwiQVdTNC1ITUFDLVNIQTI1NiJdLFsiZXEiLCIkeC1hbXotY3JlZGVudGlhbCIsIkFLSUFKVkE1Qk1NVTJSSE82SU9RLzIwMTUxMTA5L3VzLWVhc3QtMS9zMy9hd3M0X3JlcXVlc3QiXV19 -F x-amz-algorithm=AWS4-HMAC-SHA256 -F x-amz-credential=AKIAJVA5BMMU2RHO6IOQ/20151109/us-east-1/s3/aws4_request -F x-amz-date=20151109T061321Z -F key=2007-Mar-2/<FILE> -F file=@<FILE> ",
  "keyName": "https://s3.amazonaws.com/backup/2007-Mar-2/..."
}

期待一些反馈。谢谢

Answer 6

知道这已经晚了 8 个月，但我刚刚发现您可以使用 Eclipse 和 AWS Toolkit for Eclipse ( http://aws.amazon.com/eclipse/ ) 通过右键单击目标生成预签名 URL文件，甚至可以让您设置到期日期....它非常漂亮。

Answer 7

我认为没有任何预先构建的 cli 命令 \sub 命令来获取私有存储桶 url，对于预签名，您可以参考presign。如果您想要私有存储桶中的公共 url，那就是对象读取是公共的而不是存储桶。对于公共 aws url，你可以使用一些东西 - 我已经在 powershell 中写了这个，你可以把这个$profile

Function getAwsUrl{
[CmdletBinding(DefaultParameterSetName="S3Path")]
    param(
    [parameter (ParameterSetName="S3Path",mandatory=$true)]$s3path,
    [parameter (ParameterSetName="BucketKey")]$bucket,
    [parameter (ParameterSetName="BucketKey",mandatory=$true)]$key 
    )
if($s3path)
{
$splits = $s3path.Split('//')
$bucket =$splits[2]
$key = $splits[3]
}
else 
{
#user input bucket and key
$bucket =$bucket
$key = $key
}

#using standard aws region "us-west-2"
Write-Output "https://$bucket.s3.us-west-2.amazonaws.com/$key"
}

用法示例：确保您--acl作为公开阅读通过

aws s3 cp C:\temp\dumy.txt s3://fpd-uploads/ --acl public-read
getAwsUrl -s3path s3://fpd-uploads/dumy.txt

获取帮助 getAwsUrl

NAME
    getAwsUrl

SYNTAX
    getAwsUrl -s3path <Object>  [<CommonParameters>]

    getAwsUrl -key <Object> [-bucket <Object>]  [<CommonParameters>]


ALIASES
    None


REMARKS
    None