18

I'm using Imagine via the LIIPImagineBundle for Symfony2 to create cached versions of images stored in S3.

Cached images are stored in an S3 web enabled bucket served by CloudFront. However, the default LIIPImagineBundle implementation of S3 is far too slow for me (checking if the file exists on S3 then creating a URL either to the cached file or to the resolve functionality), so I've worked out my own workflow:

  1. Pass client the cloudfront URL where the cached image should exist
  2. Client requests the image via the cloudfront URL, if it does not exist then the S3 bucket has a redirect rule which 302 redirects the user to an imagine webserver path which generates the cached version of the file and saves it to the appropriate location on S3
  3. The webserve 301 redirects the user back to the cloudfront URL where the image is now stored and the client is served the image.

This is working fine as long as I don't use cloudfront. The problem appears to be that cloudfront is caching the 302 redirect response (even though the http spec states that they shouldn't). Thus, if I use cloudfront, the client is sent in an endless redirect loop back and forth from webserver to cloudfront, and every subsequent request to the file still redirects to the webserver even after the file has been generated.

If I use S3 directly instead of cloudfront there are no issues and this solution is solid.

According to Amazon's documentation S3 redirect rules don't allow me to specify custom headers (to set cache-control headers or the like), and I don't believe that CloudFront allows me to control the caching of redirects (if they do it's well hidden). CloudFront's invalidation options are so limited that I don't think they will work (can only invalidate 3 objects at any time)...I could pass an argument back to cloudfront on the first redirect (from the Imagine webserver) to fix the endless redirect (eg image.jpg?1), but subsequent requests to the same object will still 302 to the webserver then 301 back to cloudfront even though it exists. I feel like there should be an elegant solution to this problem but it's eluding me. Any help would be appreciated!!

4

3 回答 3

20

我通过将 CloudFront“缓存行为”设置中的“默认 TTL”设置为 来解决同样的问题0，但仍然允许通过将CacheControlS3 文件上的元数据设置为max-age=12313213.

这种方式不会缓存重定向（默认 TTL 行为），但我调整大小的图像将是（CacheControl max-age on s3 cache hit）。

于 2016-12-23T00:28:22.280 回答
1

如果你真的需要在这里使用 CloudFront，我唯一能想到的就是你不要直接让用户接受 302、301 的舞蹈。你能在前面的 S3 和整个过程中介绍某种代理脚本/页面吗？（或者这样做会破坏这一点）。

所以缓存未命中看起来像这样：

  • 访问者通过 Cloudfront 请求代理页面。
  • 代理页面从 S3 请求图像
  • 代理页面从 S3 接收到 302，按照这个到 Imagine 网络服务器
  • 理想情况下只是从这里返回图像（同时让它更新 S3），或者按照 301 返回 S3
  • 代理页面将图像返回给访问者
  • 图片由 Cloudfront 缓存
于 2015-03-13T20:44:36.020 回答
1

TL;DR：利用Lambda@Edge

我们使用 LiipImagineBundle 也面临同样的问题。

对于开发，NGINX 提供来自本地文件系统的内容，并使用简单的 proxy_pass 解析尚未存储的图像：

location ~ ^/files/cache/media/ {
    try_files $uri @public_cache_fallback;
}

location @public_cache_fallback {
    rewrite ^/files/cache/media/(.*)$ media/image-filter/$1 break;
    proxy_set_header X-Original-Host $http_host;
    proxy_set_header X-Original-Scheme $scheme;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_pass http://0.0.0.0:80/$uri;
}

一旦您想集成 CloudFront，由于缓存，事情会变得更加复杂。虽然您可以轻松地将 S3（静态网站，见下文）添加为分配，但 CloudFront 本身不会遵循生成的重定向，而是将它们返回给客户端。在默认配置中，CloudFront 将缓存此重定向而不是所需的图像（有关S3 的解决方法，请参阅https://stackoverflow.com/a/41293603/6669161 ）。

最好的方法是使用这里描述的代理。然而，这增加了另一个可能不受欢迎的层。另一种解决方案是使用 Lambda@Edge 函数作为（见这里）。在我们的例子中，我们使用 S3 作为正态分布并利用“原始响应”事件（您可以在分布的“行为”选项卡中编辑它们）。我们的 Lambda 函数只是检查对 S3 的请求是否成功。如果是，我们可以转发它。如果不是，我们假设尚未创建所需的对象。然后 lambda 函数调用我们的应用程序，该应用程序生成对象并将其存储在 S3 中。为简单起见，应用程序也使用重定向（再次到 CloudFront）回复 - 因此我们可以将其转发给客户端。一个缺点是客户端本身会看到一个重定向。还要确保设置缓存标头，以便 CloudFront 不会缓存 lambda 重定向。

在此处输入图像描述

这是一个示例 Lambda 函数。这只是将客户端重定向到解析 url（然后再次重定向到 CloudFront）。请记住，这将导致客户端的往返次数更多（这并不完美）。但是，它会减少 Lambda 函数的执行时间。确保添加 Base Lambda@Edge 策略（相关教程）。

env = {
    'Protocol': 'http',
    'HostName': 'localhost:8000',
    'HttpErrorCodeReturnedEquals': '404',
    'HttpRedirectCode': '307',
    'KeyPrefixEquals': '/cache/media/',
    'ReplaceKeyPrefixWith': '/media/resolve-image-filter/'
}


def lambda_handler(event, context):
    response = event['Records'][0]['cf']['response']
    
    if int(response['status']) == int(env['HttpErrorCodeReturnedEquals']):
        request = event['Records'][0]['cf']['request']
        original_path = request['uri']
        
        if original_path.startswith(env['KeyPrefixEquals']):
            new_path = env['ReplaceKeyPrefixWith'] + original_path[len(env['KeyPrefixEquals']):]
        else:
            new_path = original_path
            
        location = '{}://{}{}'.format(env['Protocol'], env['HostName'], new_path)
        
        response['status'] = env['HttpRedirectCode']
        response['statusDescription'] = 'Resolve Image'
        response['headers']['location'] = [{
                 'key': 'Location',
                 'value': location
             }]
        response['headers']['cache-control'] = [{
                 'key': 'Cache-Control',
                 'value': 'no-cache'    # Also make sure that you minimum TTL is set to 0 (for the distribution)
             }]
        
    return response

如果您只想将 S3 用作缓存（没有 CloudFront）。如果缺少缓存文件，使用静态网站托管和重定向规则会将客户端重定向到解析 url（尽管您需要将 S3 Cache Resolver url 重写为网站版本）：

 <RoutingRules>
  <RoutingRule>
    <Condition><HttpErrorCodeReturnedEquals>403</HttpErrorCodeReturnedEquals>
      <KeyPrefixEquals>cache/media/</KeyPrefixEquals>
    </Condition>
    <Redirect>
      <Protocol>http</Protocol>
      <HostName>localhost</HostName>
      <ReplaceKeyPrefixWith>media/image-filter/</ReplaceKeyPrefixWith>
      <HttpRedirectCode>307</HttpRedirectCode>
    </Redirect>
  </RoutingRule>
</RoutingRules>
于 2020-10-04T14:40:44.097 回答