20

如何使用 PHP API 将包含子文件夹和文件的文件夹复制/复制到 S3 存储桶中的另一个目录中？

$s3->copy_object只复制文件夹，而不复制里面的文件和子文件夹。

我是否必须使用$s3->list_objects来获取所有文件和目录并$s3->copy_object在每个文件/目录上运行？

4

6 回答 6

18

S3 不是文件系统，它是对象存储。文件夹实际上并不存在于任何有形的意义上。文件夹只是您可以称为共享前缀的东西。换句话说，如果你创建path/to/oneand path/to/two，它也不会导致pathandpath/to存在。如果您看到它们，那是因为某些组件获取了一个对象列表，将它们的键拆分为/，并决定将该列表显示为层次结构。

您想“将一个文件夹复制到另一个文件夹中”。将其重新表述为 S3 术语，您希望“将具有相同前缀的所有对象复制到具有不同前缀的对象中”。这么说使方法变得清晰：获取带有一个前缀的对象列表，然后复制它们中的每一个。

于 2012-09-21T02:16:50.993 回答
3

scala 的代码（在一个存储桶中的文件夹之间复制）：

def copyFolders(bucketName: String, srcFolder: String, targetFolder: String): Unit = {
import scala.collection.JavaConversions._
val transferManager: TransferManager = TransferManagerBuilder.standard.build
try {

  for (file <- s3.listObjects(bucketName, s"$srcFolder/").getObjectSummaries) {
    val fileName = file.getKey.replace(s"$srcFolder/", "")
    if (!fileName.isEmpty) {
      val transferProcess: Copy = transferManager.copy(bucketName, file.getKey,
        bucketName, s"$targetFolder/$fileName")
      log.info(s"Old key = ${file.getKey}")
      log.info(s"New file Key = $targetFolder/$fileName")
      transferProcess.waitForCompletion()
    }
  }
} catch {
  case e: AmazonServiceException =>
    log.error(e.getErrorMessage, e)
    System.exit(1)
  case e: AmazonClientException =>
    log.error("Amazon client error: " + e.getMessage, e)
    System.exit(1)
  case e: InterruptedException =>
    log.error("Transfer interrupted: " + e.getMessage, e)
    System.exit(1)
}
}

用法：

copyFolders("mybucket", "somefolder/srcfolder", "somefolder/targetfolder")
于 2019-06-10T16:58:42.433 回答
0

这是从亚马逊获取的一些代码。此代码将项目 a 三次复制到目标，您需要做的是更改它，以便它循环遍历每个键并将其添加到批处理中。

<?php

// Include the AWS SDK using the Composer autoloader.
require 'vendor/autoload.php';

use Aws\S3\S3Client;

$sourceBucket = '*** Your Source Bucket Name ***';
$sourceKeyname = '*** Your Source Object Key ***';
$targetBucket = '*** Your Target Bucket Name ***';

// Instantiate the client.
$s3 = S3Client::factory();

// Copy an object.
$s3->copyObject(array(
    'Bucket'     => $targetBucket,
    'Key'        => "{$sourceKeyname}-copy",
    'CopySource' => "{$sourceBucket}/{$sourceKeyname}",
));

// Perform a batch of CopyObject operations.
$batch = array();
for ($i = 1; $i <= 3; $i++) {
    $batch[] = $s3->getCommand('CopyObject', array(
        'Bucket'     => $targetBucket,
        'Key'        => "{$sourceKeyname}-copy-{$i}",
        'CopySource' => "{$sourceBucket}/{$sourceKeyname}",
    ));
}
try {
    $successful = $s3->execute($batch);
    $failed = array();
} catch (\Guzzle\Service\Exception\CommandTransferException $e) {
    $successful = $e->getSuccessfulCommands();
    $failed = $e->getFailedCommands();
}
于 2014-06-04T20:18:17.350 回答
0

一种方法是使用列表对象并逐个移动每个对象。另一种方法是使用 s3fuse，它将您的 s3 存储桶作为本地目录，然后您可以应用简单的命令（如“mv”）来移动文件。

于 2012-09-21T00:00:07.943 回答
0

对于那些使用 aws sdk 寻找 java 解决方案的人：

public static void copyAllFilesWithPrefix(AmazonS3 s3Client,
                                          String srcBucket,
                                          String srcPrefix,
                                          String destBucket,
                                          String destPrefix) {
    ObjectListing objectListing = s3Client.listObjects(srcBucket, srcPrefix);
    for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
        S3Object srcFile = s3Client.getObject(srcBucket, objectSummary.getKey());
        String destKey = srcFile.getKey().replace(srcPrefix, destPrefix);
        if (!s3Client.doesObjectExist(destBucket, destKey)) {
            copyFile(s3Client, srcFile, destBucket, srcFile.getKey().replace(srcPrefix, destPrefix));
        }
    }
}
于 2020-12-15T13:01:09.010 回答
0

使用@aws-sdk/client-s3，这里是完整的 js 代码：

import {
    S3Client,
    CopyObjectCommandInput,
    CopyObjectCommand,
    ListObjectsCommandInput,
    ListObjectsCommand
} from '@aws-sdk/client-s3';
  
const s3Client = new S3Client({ region: 'us-east-1' });

const listS3Objects = async (input: ListObjectsCommandInput) => {
     const command = new ListObjectsCommand(input);
     return await s3Client.send(command);
}

const copyS3Object = async (input: any) => {
    const command = new CopyObjectCommand(input);
    return await s3Client.send(command);
};

export const copyFolder = async (fromBucket:string, fromFolderKey:string, toBucket:string, toFolderKey: any) => {
      
try {
    const listObjectsResponse = await listS3Objects({
        Bucket: fromBucket,
        Prefix: `${fromFolderKey}/`
    })

    const fromObjectKeys = listObjectsResponse.Contents.map(content => content.Key)

    for (let fromObjectKey of fromObjectKeys) {
        const toObjectKey = fromObjectKey.replace(fromFolderKey, toFolderKey);

       await copyS3Object({
          Bucket: toBucket,
          CopySource: `${fromBucket}/${fromObjectKey}`,
          Key: toObjectKey,
       })
    }
  } catch (error) {
     throw new Error(error)
  }
}
于 2021-08-19T21:03:03.597 回答