2

我有一个自动例程，可由用户触发以将图像上传到 Amazon S3。用户通常会上传超过 500 个项目，我正在努力寻找一种方法来避免进程超时。

现在我正在这样做：

<form action="hs_import.cfm?ansicht=Bilder&RequestTimeout=5000" method="post" name="uploader">
...
    <input type="button" OnClick="bilder_upload()" value="#tx_gen_run#">
    <input type="hidden" name="artikel_uploaden" value="ja">
    <input type="hidden" name="ansicht" value="imageloader">
</form>

这触发了一个javascript函数，它触发了我的上传（没有详细信息）：

<cfif isdefined("artikel_uploaden")>
<cfscript>
        S3 variables
</cfscript>

    <!--- get img paths to upload --->
    <cfquery datasource="db" name="img_paths">
    SELECT DISTINCT imgpath
</cfquery>

    <cfif img_paths.recordcount GT 0>
        <cfloop query="img_paths">
            <cfif img_paths.typ NEQ "img">
                <cfset variables.testFilePath = img_paths.bildpfad & img_paths.bilddateiname>
                <cfset variables.fileExt = ListLast(variables.testFilePath, ".")>


                <!--- get image --->
                <cfhttp timeout="45" 
                    throwonerror="no" 
                    url="#variables.testFilePath#" 
                    method="get" 
                    useragent="Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" 
                    getasbinary="yes" 
                    result="variables.objGet">

                <!--- validate --->
                <!--- upload 4 sizes (s,m,l,xl) to S3 --->          
                <cftry>
                    <cfset objImage = ImageNew(variables.objGet.FileContent )>
                    <cfimage source="#objImage#" action="write" quality=".99" destination="#variables.tempDirectory#_base_#img_paths.bilddateiname#" overwrite="yes">
                    <cfset variables.basePath = variables.tempDirectory & "_base_" & img_paths.bilddateiname>
                    <cfimage action="read" source="#variables.basePath#" name="base">
                    <cfset variables.imageSrc = variables.tempDirectory>
                    <cfscript>
                        if ( ImageGetWidth( base ) LT ImageGetHeight( base ) ) {
                        // portrait                                     
                        } else {
                        // landscape/square

                        }
                        // cleanup
                    </cfscript>

                    <!--- create IMG entry in media table  --->
                    <cfquery datasource="db"></cfquery>
                </cfif>
            <cfcatch>
                <cfset variables.errorCount = variables.errorCount+1>
                <cfset variables.failedLoads = variables.failedLoads & img_paths.bilddateiname & "  (" & tx_pop_error & ":" & tx_errors_import_ext & "), ">
            </cfcatch>
        </cftry>
    </cfif>
  </cfloop>
  <!--- alert on success and errors --->
</cfif>

这可以正常工作，但是如果图像数量太大，浏览器就会挂起/超时，所以我通常会看到一个加载屏幕，它永远不会完成并且不会提醒成功/错误。

问题：
处理这样的大文件上传有什么更好的选择？例如，这是否应该进入 a cfschedule，以便它可以在后台运行？

感谢提示！

4

3 回答 3

2

You're trying to do an awful lot in a single request. While one or five images may not generate timeouts, hundreds of messages certainly will. From the code you provided above, you're doing the following in a single request:

  1. Upload [n] images
  2. Make an http request to fetch each image
  3. Resize each image (possibly into 4 different sizes, not entirely clear from the sample)
  4. Upload each manipulated version of each image to S3

That's a lot of work, especially given that both cfhttp and, more specifically, cfimage are not the fastest tags in the language. Image manipulation via cfimage can be quite slow.

You're probably much better off doing what you suggested: breaking these tasks apart and doing as much of the processing on the backend so that the original customer request doesn't have to wait. You could do the initial image upload and then end the customer request, notifying them that it may take a few minutes for all of their images to be processed.

You then have the option of using a scheduled task or a task queue (ie; Rabbit MQ) to do the image processing and uploading to S3. I'd even suggest that you have a separate instance which handles scheduled tasks, so that this processing work isn't done on the instances where customers are interacting directly with your web app. Breaking the image manipulation and S3 upload into separate tasks might even be better, so that you can have error handling to capture problems with image manipulation (ie; bad file formats) without also borking the S3 upload process. This requires a bit more tracking work on the backend (ie; which images still need processing, which images still need uploading to S3) but this isn't hard to implement and gives you a more robust workflow setup overall.

于 2012-11-06T14:24:12.020 回答
0

如果您希望用户触发请求，您可以通过 AJAX 调用来实现，而无需等待回复。只需告诉用户请求正在后台运行。您还可以在完成页面线程并显示“正在处理的请求”页面时使用 cfthread 启动大请求。

问 Ben：在 ColdFusion 中使用 CFThread 处理文件

于 2012-11-06T17:53:52.897 回答
0

您正在循环执行 cfquery，这样做非常糟糕，并且可能导致严重的减速。解决此问题的一种方法是将 100 个查询存储到一个变量中，然后一次运行多个查询（连接的打开和关闭可能需要很长时间。）当我使用这种样式进行大规模更新时，会得到性能提高了 90%。

这可能足以让您的流程正常工作。当然，如果你有很多非常大的图像，那也没什么可做的（限制图像大小除外）

于 2012-11-06T14:21:10.933 回答