1

我想到了

这是缺失的部分。一旦我清理了我的代码，我会发布一个答案，这样希望下一个必须处理这个问题的可怜的灵魂不必经历我经历过的同样的地狱；）

$command = $client->getCommand('UploadPart', array(
    'Bucket' => 'the-bucket-name',
    'Key' => $key,
    'PartNumber' => $partNumber,
    'UploadId' => $uploadId,
    'Body' => '',
));

$signedUrl = $client->createPresignedRequest($command, '+20 minutes');
$presignedUrl = (string)$signedUrl->getUri();
return response()->json(['url' => $presignedUrl]);

我试图弄清楚如何配置我的服务器以使用 Uppy 使用 CompanionUrl 选项将分段上传上传到 AWS S3。https://uppy.io/docs/aws-s3-multipart/#createMultipartUpload-file。

这就是我想走这条路的地方https://github.com/transloadit/uppy/issues/1189#issuecomment-445521442。

我无法弄清楚这一点，我觉得其他人也被卡住了，没有答案，所以我发布了到目前为止我想出的东西，试图让 Uppy 使用 Laravel/Vue 处理分段上传。

对于 Vue 组件，我有这个：

<template>
<div>
    <a id="uppy-trigger" @click="isUppyOpen = !isUppyOpen">Open Uppy</a>

    <dashboard-modal
        :uppy="uppy"
        :open="isUppyOpen"
        :props="{trigger: '#uppy-trigger'}"
    />
</div>
</template>

<script>
import Uppy from '@uppy/core'
import AwsS3Multipart from '@uppy/aws-s3-multipart';
import '@uppy/core/dist/style.css';
import '@uppy/dashboard/dist/style.css';

export default {
    components: {
        'dashboard-modal': DashboardModal,
    },

    data() {
        return {
            isUppyOpen: false,
        }
    },

    computed: {
        // Uppy Instance
        uppy: () => new Uppy({
            logger: Uppy.debugLogger
        }).use(AwsS3Multipart, {
            limit: 4,
            companionUrl: 'https://mysite.local/',
        }),
    },

    beforeDestroy () {
        this.uppy.close();
    },
}
</script>

然后对于路由，我已将其添加到我的 web.php 文件中。

// AWS S3 Multipart Upload Routes
Route::name('s3.multipart.')->prefix('s3/multipart')
    ->group(function () {
        Route::post('/', ['as' => 'createMultipartUpload', 'uses' => 'AwsS3MultipartController@createMultipartUpload']);
        Route::get('{uploadId}', ['as' => 'getUploadedParts', 'uses' => 'AwsS3MultipartController@getUploadedParts']);
        Route::get('{uploadId}/{partNumber}', ['as' => 'signPartUpload', 'uses' => 'AwsS3MultipartController@signPartUpload']);
        Route::post('{uploadId}/complete', ['as' => 'completeMultipartUpload', 'uses' => 'AwsS3MultipartController@completeMultipartUpload']);
        Route::delete('{uploadId}', ['as' => 'abortMultipartUpload', 'uses' => 'AwsS3MultipartController@abortMultipartUpload']);
    });

基本上发生的事情是我将“companionUrl”设置为“https://mysite.local/”，然后Uppy将在将分段上传文件上传到这些路由时发送五个请求，即“https://mysite.local”。本地/s3/multipart/createMultipartUpload”。

然后我创建了一个控制器来处理请求：

<?php

namespace App\Http\Controllers;

use Aws\S3\S3Client;
use Illuminate\Http\Request;

class AwsS3MultipartController extends Controller
{
    public function createMultipartUpload(Request $request)
    {
        $client = new S3Client([
            'version' => 'latest',
            'region'  => 'us-east-1',
        ]);

        $key = $request->has('filename') ? $request->get('filename') : null;

        $type = $request->has('type') ? $request->get('type') : null;

        if (!is_string($key)) {
            return response()->json(['error' => 's3: filename returned from "getKey" must be a string'], 500);
        }

        if (!is_string($type)) {
            return response()->json(['error' => 's3: content type must be a string'], 400);
        }

        $response = $client->createMultipartUpload([
            'Bucket'        => 'the-bucket-name',
            'Key'           => $key,
            'ContentType'   => $type,
            'Expires'       => 60
        ]);

        $mpuKey = !empty($response['Key']) ? $response['Key'] : null;
        $mpuUploadId = !empty($response['UploadId']) ? $response['UploadId'] : null;

        if (!$mpuKey || !$mpuUploadId) {
            return response()->json(['error' => 'Unable to process upload request.'], 400);
        }

        return response()->json([
            'key'       => $mpuKey,
            'uploadId'  => $mpuUploadId
        ]);
    }

    public function getUploadedParts($uploadId)
    {
        // Haven't configured this route yet as I haven't made it this far.
        return $uploadId;
    }

    public function signPartUpload(Request $request, $uploadId, $partNumber)
    {
        $client = new S3Client([
            'version' => 'latest',
            'region'  => 'us-east-1',
        ]);

        $key = $request->has('key') ? $request->get('key') : null;

        if (!is_string($key)) {
            return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
        }

        if (!intval($partNumber)) {
            return response()->json(['error' => 's3: the part number must be a number between 1 and 10000.'], 400);
        }

        // Creating a presigned URL. I don't think this is correct.
        $cmd = $client->getCommand('PutObject', [
            'Bucket'        => 'the-bucket-name',
            'Key'           => $key,
            'UploadId'      => $uploadId,
            'PartNumber'    => $partNumber,
        ]);

        $response = $client->createPresignedRequest($cmd, '+20 minutes');
        $presignedUrl = (string)$response->getUri();

        return response()->json(['url' => $presignedUrl]);
    }

    public function completeMultipartUpload(Request $request, $uploadId)
    {
        $client = new S3Client([
            'version' => 'latest',
            'region'  => 'us-east-1',
        ]);

        $key = $request->has('key') ? $request->get('key') : null;

        $parts = json_decode($request->getContent(), true)['parts'];

        if (!is_string($key)) {
            return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
        }

        if (!is_array($parts) || !$this->arePartsValid($parts)) {
            return response()->json(['error' => 's3: "parts" must be an array of {ETag, PartNumber} objects.'], 400);
        }


        // The completeMultipartUpload method fails with the following error.

        // "Error executing "CompleteMultipartUpload" on "https://the-bucket-name.s3.amazonaws.com/NewProject.png?uploadId=nlWLdbNgB9zgarpLBXnj17eOIGAmQM_xyBArymtwdM71fhbFvveggDmL6fz4blz.B95TLhMatDvodbMb5p2ZMKqdlLeLFoSW1qcu33aRQTlt6NbiP_dkDO90DFO.pWGH"; AWS HTTP error: Client error: `POST https://the-bucket-name.s3.amazonaws.com/NewProject.png?uploadId=nlWLdbNgB9zgarpLBXnj17eOIGAmQM_xyBArymtwdM71fhbFvveggDmL6fz4blz.B95TLhMatDvodbMb5p2ZMKqdlLeLFoSW1qcu33aRQTlt6NbiP_dkDO90DFO.pWGH` resulted in a `400 Bad Request` response:
        //     <Error><Code>InvalidPart</Code><Message>One or more of the specified parts could not be found.  The part may not have be (truncated...)
        //  InvalidPart (client): One or more of the specified parts could not be found.  The part may not have been uploaded, or the specified entity tag may not match the part's entity tag. - <Error><Code>InvalidPart</Code><Message>One or more of the specified parts could not be found.  The part may not have been uploaded, or the specified entity tag may not match the part's en"

        $result = $client->completeMultipartUpload([
            'Bucket'          => 'the-bucket-name',
            'Key'             => $key,
            'UploadId'        => $uploadId,
            'MultipartUpload' => [
                'Parts' => $parts,
            ],
        ]);

        return response()->json(['location' => $result['location']]);
    }

    public function abortMultipartUpload($uploadId)
    {
        // Haven't configured this route yet as I haven't made it this far.
        return $uploadId;
    }

    private function arePartsValid($parts)
    {
        // Validation for the parts will go here, but returning true for now.
        return true;
    }
}

我可以上传一个纯 PHP/服务器端的多部分文件。但是，对于大文件，这不起作用，因为我必须等待上传在我的服务器上完成，然后将其上传到各个部分的 AWS。

$s3_client = new S3Client([
    'version' => 'latest',
    'region'  => 'us-east-1',
]);
$bucket = 'the-bucket-name';
$tmp_name = $request->file('file')->getPathname();
$folder = Carbon::now()->format('Y/m/d/');
$filename = pathinfo($request->file('file')->getClientOriginalName(), PATHINFO_FILENAME);
$extension = $extension = pathinfo($request->file('file')->getClientOriginalName(), PATHINFO_EXTENSION);
$timestamp = Carbon::now()->format('H-i-s');
$name = "{$folder}{$filename}_{$timestamp}.{$extension}";

$response = $s3_client->createMultipartUpload([
    'Bucket' => $bucket,
    'Key'    => $name,
]);

$uploadId = $response['UploadId'];

$file = fopen($tmp_name, 'r');
$parts = [];
$partNumber = 1;
while (! feof($file)) {
    $result = $s3_client->uploadPart([
        'Bucket'     => $bucket,
        'Key'        => $name,
        'UploadId'   => $uploadId,
        'PartNumber' => $partNumber,
        'Body'       => fread($file, 5 * 1024 * 1024),
    ]);

    $parts[] = [
        'PartNumber' => $partNumber++,
        'ETag'       => $result['ETag'],
    ];
}

$result = $s3_client->completeMultipartUpload([
    'Bucket'          => $bucket,
    'Key'             => $name,
    'UploadId'        => $uploadId,
    'MultipartUpload' => [
        'Parts' => $parts,
    ],
]);

我认为正在发生的是 Uppy 正在处理while客户端的循环部分。为此，我必须返回 Uppy 可以使用的预签名 URL，但我当前返回的预签名 URL 不正确。

我注意到的一件事是，当我在纯服务器端启动分段上传时单步执行 while 循环时，在触发completeMultipartUpload方法之前，没有文件上传到我的存储桶。但是，如果我逐步检查通过 Uppy 上传的部分，则这些部分似乎是作为最终文件上传的，并且每个部分只是覆盖了前一部分。然后我留下了文件的一个片段，即 43.5MB 文件的最后 3.5MB。

4

2 回答 2

2

以下是我如何让 Uppy、Vue 和 Laravel 很好地协同工作。

Vue 组件：

<template>
<div>
    <a id="uppy-trigger" @click="isUppyOpen = !isUppyOpen">Open Uppy</a>

    <dashboard-modal
        :uppy="uppy"
        :open="isUppyOpen"
        :props="{trigger: '#uppy-trigger'}"
    />
</div>
</template>

<script>
import Uppy from '@uppy/core'
import AwsS3Multipart from '@uppy/aws-s3-multipart';
import '@uppy/core/dist/style.css';
import '@uppy/dashboard/dist/style.css';

export default {
    components: {
        'dashboard-modal': DashboardModal,
    },

    data() {
        return {
            isUppyOpen: false,
        }
    },

    computed: {
        // Uppy Instance
        uppy: () => new Uppy({
            logger: Uppy.debugLogger
        }).use(AwsS3Multipart, {
            limit: 4,
            companionUrl: 'https://mysite.local/',
        }),
    },

    beforeDestroy () {
        this.uppy.close();
    },
}
</script>

路由：

// AWS S3 Multipart Upload Routes
Route::name('s3.multipart.')->prefix('s3/multipart')
    ->group(function () {
        Route::post('/', ['as' => 'createMultipartUpload', 'uses' => 'AwsS3MultipartController@createMultipartUpload']);
        Route::get('{uploadId}', ['as' => 'getUploadedParts', 'uses' => 'AwsS3MultipartController@getUploadedParts']);
        Route::get('{uploadId}/{partNumber}', ['as' => 'signPartUpload', 'uses' => 'AwsS3MultipartController@signPartUpload']);
        Route::post('{uploadId}/complete', ['as' => 'completeMultipartUpload', 'uses' => 'AwsS3MultipartController@completeMultipartUpload']);
        Route::delete('{uploadId}', ['as' => 'abortMultipartUpload', 'uses' => 'AwsS3MultipartController@abortMultipartUpload']);
    });

控制器：

<?php

namespace App\Http\Controllers;

use Aws\S3\S3Client;
use Carbon\Carbon;
use Exception;
use Illuminate\Http\Request;

class AwsS3MultipartController extends Controller
{
    private $bucket;
    private $client;

    public function __construct()
    {
        $this->bucket = 'the-name-of-the-bucket';

        $this->client = new S3Client([
            'version' => 'latest',
            'region'  => 'us-east-1',
        ]);
    }

    /**
     * Create/initiate the multipart upload
     * @param Request $request 
     * @return JsonResponse 
     */
    public function createMultipartUpload(Request $request)
    {
        // Get the filename and type from request
        $filename = $request->has('filename') ? $request->get('filename') : null;
        $type = $request->has('type') ? $request->get('type') : null;

        // Check filename
        if (!is_string($filename)) {
            return response()->json(['error' => 's3: filename returned from "getKey" must be a string'], 500);
        }

        // Check type
        if (!is_string($type)) {
            return response()->json(['error' => 's3: content type must be a string'], 400);
        }

        // Set up key equal to YYYY/MM/DD/filename_H-i-s.ext
        $fileBaseName = pathinfo($filename, PATHINFO_FILENAME);
        $extension = pathinfo($filename, PATHINFO_EXTENSION);
        $folder = Carbon::now()->format('Y/m/d/');
        $timestamp = Carbon::now()->format('H-i-s');
        $key = "{$folder}{$fileBaseName}_{$timestamp}.{$extension}";

        // Create/initiate the multipart upload
        try {
            $response = $this->client->createMultipartUpload([
                'Bucket'        => $this->bucket,
                'Key'           => $key,
                'ContentType'   => $type,
                'Expires'       => 60
            ]);
        } catch (Exception $e) {
            return response()->json(['error' => $e->getMessage()], 400);
        }

        // Multipart upload key and id
        $mpuKey = !empty($response['Key']) ? $response['Key'] : null;
        $mpuUploadId = !empty($response['UploadId']) ? $response['UploadId'] : null;

        // Check multipart upload key and id
        if (!$mpuKey || !$mpuUploadId) {
            return response()->json(['error' => 'Unable to process upload request.'], 400);
        }

        return response()->json([
            'key'       => $mpuKey,
            'uploadId'  => $mpuUploadId
        ]);
    }

    /**
     * Get parts that have been uploaded
     * @param Request $request 
     * @param string $uploadId 
     * @return JsonResponse 
     */
    public function getUploadedParts(Request $request, string $uploadId)
    {
        $key = $request->has('key') ? $request->get('key') : null;

        // Check key
        if (!is_string($key)) {
            return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
        }

        $parts = [];
        $getParts = true;
        $startAt = 0;

        // Get parts uploaded so far
        while ($getParts) {
            $partsPage = $this->listPartsPage($key, $uploadId, $startAt, $parts);

            if (isset($partsPage['error'])) {
                return response()->json(['error' => $partsPage['error']], 400);
            }

            if ($partsPage['isTruncated']) {
                $startAt = $partsPage['nextPartNumberMarker'];
            } else {
                $getParts = false;
            }
        }

        return response()->json(
            $parts,
        );
    }

    /**
     * Create a pre-signed URL for parts to be uploaded to
     * @param Request $request 
     * @param string $uploadId 
     * @param int $partNumber 
     * @return JsonResponse 
     */
    public function signPartUpload(Request $request, string $uploadId, int $partNumber)
    {
        $key = $request->has('key') ? $request->get('key') : null;

        // Check key
        if (!is_string($key)) {
            return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
        }

        // Check part number
        if (!intval($partNumber)) {
            return response()->json(['error' => 's3: the part number must be a number between 1 and 10000.'], 400);
        }

        // Create the upload part command and get the pre-signed URL
        try {
            $command = $this->client->getCommand('UploadPart', [
                'Bucket'        => $this->bucket,
                'Key'           => $key,
                'PartNumber'    => $partNumber,
                'UploadId'      => $uploadId,
                'Body'          => '',
            ]);

            $presignedUrl = $this->client->createPresignedRequest($command, '+20 minutes');
        } catch (Exception $e) {
            return response()->json(['error' => $e->getMessage()], 400);
        }

        // Convert the pre-signed URL to a string
        $presignedUrlString = (string)$presignedUrl->getUri();

        return response()->json(['url' => $presignedUrlString]);
    }

    /**
     * Complete the multipart upload
     * @param Request $request 
     * @param string $uploadId 
     * @return JsonResponse 
     */
    public function completeMultipartUpload(Request $request, string $uploadId)
    {
        $key = $request->has('key') ? $request->get('key') : null;

        $parts = json_decode($request->getContent(), true)['parts'];

        // Check the key
        if (!is_string($key)) {
            return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
        }

        // Check the parts
        if (!is_array($parts) || !$this->arePartsValid($parts)) {
            return response()->json(['error' => 's3: "parts" must be an array of {ETag, PartNumber} objects.'], 400);
        }

        // Complete the multipart upload
        try {
            $result = $this->client->completeMultipartUpload([
                'Bucket'          => $this->bucket,
                'Key'             => $key,
                'UploadId'        => $uploadId,
                'MultipartUpload' => [
                    'Parts' => $parts,
                ],
            ]);
        } catch (Exception $e) {
            return response()->json(['error' => $e->getMessage()], 400);
        }

        // Change forwardslash entities to forwardslashes
        $location = urldecode($result['Location']);

        return response()->json(['location' => $location]);
    }

    public function abortMultipartUpload(Request $request, $uploadId)
    {
        $key = $request->has('key') ? $request->get('key') : null;

        // Check the key
        if (!is_string($key)) {
            return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
        }

        // Cancel the multipart upload
        try {
            $response = $this->client->abortMultipartUpload([
                'Bucket' => $this->bucket,
                'Key' => $key,
                'UploadId' => $uploadId,
            ]);
        } catch (Exception $e) {
            //
        }

        return response()->json();
    }

    private function listPartsPage(string $key, string $uploadId, int $startAt, array &$parts)
    {
        // Configure response
        $response = [
            'isTruncated' => false,
        ];

        // Get list of parts uploaded
        try {
            $result = $this->client->listParts([
                'Bucket'            => $this->bucket,
                'Key'               => $key,
                'PartNumberMarker'  => $startAt,
                'UploadId'          => $uploadId,
            ]);
        } catch (Exception $e) {
            return ['error' => 's3: unable to continue upload. The upload may have been aborted.'];
        }

        // Add found parts to parts array
        if ($result->hasKey('Parts')) {
            array_push($parts, ...$result->get('Parts'));
        }

        // Check if parts are truncated
        if ($result->hasKey('IsTruncated') && $result->get('IsTruncated')) {
            $response['isTruncated'] = true;
            $response['nextPartNumberMarker'] = $result->get('NextPartNumberMarker');
        }

        return $response;
    }

    /**
     * Validate the parts for the multipart upload
     * @param array $parts An associative array of parts with PartNumber and ETag
     * @return bool 
     */
    private function arePartsValid(array $parts)
    {
        if (!is_array($parts)) {
            return false;
        }

        foreach ($parts as $part) {
            if (!is_int($part['PartNumber']) || !is_string($part['ETag'])) {
                return false;
            }
        }

        return true;
    }
}
于 2021-04-23T14:16:01.330 回答
0

您可以使用这个预先构建的 laravel 包，通过 laravel 和 uppy 轻松实现分段上传：

https://github.com/TappNetwork/laravel-uppy-s3-multipart-upload

于 2021-05-14T04:34:50.307 回答