0
  1. 我正在尝试恢复由于 Internet 故障而失败的下载。我用来检查 curl 下载是否成功的函数是:

    curl_multi_info_read
    

    CURLE_COULDNT_CONNECT当 Internet 丢失时第一次调用此函数时,它会返回正确的错误代码 ( )。如果我再次尝试调用它,它会返回 NULL 指针,这意味着没有消息。实际上,我正在使用返回错误代码来检查是否存在互联网连接。这让我很困扰,因为如果没有互联网,它在第二次调用时不会返回任何错误代码。谁能告诉我如何使用此功能检查返回码,因为此错误码(CURLE_COULDNT_CONNECT)对我检查互联网状态非常重要,因此在我恢复连接时从停止的地方恢复下载。 ...

  2. 为了恢复下载,我正在使用

    curl_easy_setopt (curl, CURLOPT_RESUME_FROM, InternalOffset);
    

    每次我失去互联网连接时,我都会调用此函数来设置选项,以便在互联网连接恢复时可以恢复下载......


丹尼尔斯坦伯格的注释:

以下是有关平台和 libcurl 版本的一些详细信息:

  • curl 版本 - libcurl 7.21.6
  • 平台 - Linux (Ubuntu)

注释:

  1. 是的。你的看法是对的。我从堆栈中删除了简单句柄,通过设置新选项 ( curl_easy_setopt(curl, CURLOPT_RESUME_FROM, InternalOffset)) 再次添加到多句柄,最后我做了多执行。如果他们没有互联网连接,它会返回正确的错误。我的问题是:每次我失去互联网连接时是否需要重复上述步骤才能得到正确的错误?如果我不执行这些步骤,curl_multi_info_read函数将始终返回 NULL。

  2. 我所做的另一项观察是,当互联网连接恢复时,下载开始恢复。它从之前停止的位置开始下载。这对我来说是一个惊喜。curl 是否在内部负责在重新连接到 Internet 时恢复下载。如果这是对的吗?我真的需要小心恢复下载或在正确处理它时离开卷曲吗?

4

1 回答 1

5

您没有明确说明您使用的是multi interface还是easy interface可能值得一提的是您正在使用什么平台以及libcurl您正在使用什么等。

下面是最小卷曲easymulti针对libcurl/7.21.6.

我很高兴地拔掉了网线,停止了 http 服务器等等~它似乎可以应付。

这些可能会帮助您:

curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, dl_lowspeed_bytes); //bytes/sec
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, dl_lowspeed_time); //seconds
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

注意:当连接断开时,您必须非常努力地摔倒 curl。这是设计使然,但令一些人感到惊讶。

编辑

我怀疑你会想要使用CURLOPT_TIMEOUT. 这将使传输超时。如果您的下载量很大,那么几乎肯定会花费比您准备等待找出网络连接是否有问题的时间更长的时间 - 超时会受到影响。相比之下,CURLOPT_LOW_SPEED_TIME即使经过数小时的传输时间,超时也可能永远不会受到影响。

curltest_easy.c:

/*----------------------------------------------------
curltest_easy.c 
WARNING: for test purposes only ~ 
*/
#include <stdio.h>
#include <unistd.h>
#include <curl/curl.h>
#include <curl/types.h>
#include <curl/easy.h>
#include <sys/stat.h>



static int dl_progress(void *clientp,double dltotal,double dlnow,double ultotal,double ulnow)
{
    if (dlnow && dltotal)
        printf("dl:%3.0f%%\r",100*dlnow/dltotal); //shenzi prog-mon 
    fflush(stdout);    
    return 0;
}

static size_t dl_write(void *buffer, size_t size, size_t nmemb, void *stream)
{    
    return fwrite(buffer, size, nmemb, (FILE*)stream); 
}


int do_dl(void) 
{
    CURL *curl;
    FILE *fp;
    CURLcode curl_retval;
    long http_response;
    double dl_size;
    int retval=0;
    long dl_lowspeed_bytes=1000; //1K
    long dl_lowspeed_time=10; //sec        
    /*put something biG here, preferably on a server that you can switch off at will ;) */
    char url[] = {"http://fc00.deviantart.net/fs26/f/2008/134/1/a/Dragon_VII_by_NegativeFeedback.swf"};
    char filename[]={"blah.dl"};

    struct stat st={0};    
    if (!stat(filename, &st));    
    printf("st.st_size:[%ld]\n", st.st_size);  


    if(!(fp=fopen(filename, "ab"))) /*append binary*/
      return 1; 


    curl_global_init(CURL_GLOBAL_DEFAULT);   
    curl = curl_easy_init();

    if (curl) 
    {   
        //http://linux.die.net/man/3/curl_easy_setopt
        curl_easy_setopt(curl, CURLOPT_URL, url);

        /*callbacks*/
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, dl_write);
        curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dl_progress);
        curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);

        /*curl will keep running -so you have the freedom to recover 
        from network disconnects etc in your own way without
        distrubing the curl task in hand. ** this is by design :p ** */ 
        //curl_easy_setopt(curl, CURLOPT_TIMEOUT, 60);          
        //curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 30);
        /*set up min download speed threshold & time endured before aborting*/
        curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, dl_lowspeed_bytes); //bytes/sec
        curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, dl_lowspeed_time); //seconds while below low spped limit before aborting


        curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
        curl_easy_setopt(curl, CURLOPT_RESUME_FROM,st.st_size);

        /*uncomment this to get curl to tell you what its up to*/
        //curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);


        if(CURLE_OK != (curl_retval=curl_easy_perform(curl)))
        {                      
            printf("curl_retval:[%d]\n", curl_retval);
            switch(curl_retval) 
            {
                //Transferred a partial file
                case CURLE_WRITE_ERROR: //can be due to a dropped connection
                break;

                //all defined in curl/curl.h 

                default: //suggest quitting on unhandled error
                retval=0;
            };    


            curl_easy_getinfo(curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &dl_size);
            printf("CURLINFO_CONTENT_LENGTH_DOWNLOAD:%f\n", dl_size);


            curl_retval=curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &http_response);

            //see: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
            printf("CURLINFO_RESPONSE_CODE:%ld\n", http_response);

            switch(http_response)
            {
            case 0: //eg connection down  from kick-off ~suggest retrying till some max limit
            break;

            case 200: //yay we at least got to our url
            break;

            case 206:
            case 416: //http://www.checkupdown.com/status/E416.html
            printf("ouch! you might want to handle this & others\n"); 

            default: //suggest quitting on an unhandled error
            retval=0;
            };            
        }
        else
        {
            printf("our work here is done ;)\n");
            retval=2;
        }


        if (fp)
            fclose(fp);

        if (curl)
            curl_easy_cleanup(curl);
    }

    printf("retval [%d]\n", retval);
    return retval;
}


int main(void) 
{
    while (!do_dl())
    {
        usleep(5000);
    }

    return 0;
}

/* notes ----

$sudo apt-get install libcurl4-gnutls-dev
$ curl-config --libs
-L/usr/lib/i386-linux-gnu -lcurl -Wl,-Bsymbolic-functions

#oook. lets do it:
$ gcc -o curltest_easy curltest_easy.c -L/usr/lib/i386-linux-gnu -lcurl -Wl,-Bsymbolic-functions
$ ./curltest
*/

curltest_multi.c:

/*----------------------------------------------------
curltest_mult1.c
WARNING: for test purposes only ~
*/
#include <stdio.h>
#include <unistd.h>
#include <curl/curl.h>
#include <curl/types.h>
#include <curl/easy.h>
#include <sys/stat.h>

typedef struct S_dl_byte_data
{
    double new_bytes_received;  //from the latest request
    double existing_filesize;
} dl_byte_data, *pdl_byte_data;

static int dl_progress(pdl_byte_data pdata,double dltotal,double dlnow,double ultotal,double ulnow)
{
    /*dltotal := hacky way of getting the Content-Length ~ less hacky would be to first
    do a HEAD request & then curl_easy_getinfo with CURLINFO_CONTENT_LENGTH_DOWNLOAD*/
    if (dltotal && dlnow)
    {
        pdata->new_bytes_received=dlnow;
        dltotal+=pdata->existing_filesize;
        dlnow+=pdata->existing_filesize;
        printf(" dl:%3.0f%% total:%.0f received:%.0f\r",100*dlnow/dltotal, dltotal, dlnow); //shenzi prog-mon
        fflush(stdout);
    }
    return 0;
}

static size_t dl_write(void *buffer, size_t size, size_t nmemb, void *stream)
{
    return fwrite(buffer, size, nmemb, (FILE*)stream);
}

////////////////////////
int do_dl(void)
{
    CURLM *multi_handle;
    CURL *curl;
    FILE *fp;
    CURLcode curl_retval;
    int retval=0;
    int handle_count=0;
    double dl_bytes_remaining, dl_bytes_received;
    dl_byte_data st_dldata={0};
    char curl_error_buf[CURL_ERROR_SIZE]={"meh"};
    long dl_lowspeed_bytes=1000, dl_lowspeed_time=10; /* 1KBs for 10 secs*/

    /*put something biG here, preferably on a server that you can switch off at will ;) */
    char url[] = {"http://fc00.deviantart.net/fs26/f/2008/134/1/a/Dragon_VII_by_NegativeFeedback.swf"};

    char outfilename[]={"blah.swf"}, filename[]={"blah.dl"};
    struct stat st={0};


    if (!(fp=fopen(filename, "ab")) || -1==fstat(fileno(fp), &st)) //append binary
      return -1;

    if (curl_global_init(CURL_GLOBAL_DEFAULT))
      return -2;

    if (!(multi_handle = curl_multi_init()))
      return -3;

    if (!(curl = curl_easy_init()))
      return -4;


    st_dldata.new_bytes_received=st_dldata.existing_filesize=st.st_size;

    //http://curl.haxx.se/libcurl/c/curl_easy_setopt.html
    curl_easy_setopt(curl, CURLOPT_URL, url);

    /*callbacks*/
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, dl_write);
    curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dl_progress);
    curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, &st_dldata);
    curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);

    /*curl will keep running -so you have the freedom to recover from network disconnects etc
    in your own way without distrubing the curl task in hand. ** this is by design :p **
    The follwoing sets up min download speed threshold & time endured before aborting*/
    curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, dl_lowspeed_bytes); //bytes/sec
    curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, dl_lowspeed_time); //seconds while below low spped limit before aborting
    //alternatively these are available in libcurl 7.25
    //curl_easy_setopt(curl, CURLOPT_TCP_KEEPALIVE,1L);
    //curl_easy_setopt(curl, CURLOPT_TCP_KEEPIDLE,10);
    //curl_easy_setopt(curl, CURLOPT_TCP_KEEPINTVL,10);

    curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);

    /*uncomment this to get curl to tell you what its up to*/
    //curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, curl_error_buf);


    do
    {
        if (st_dldata.new_bytes_received) //set the new range for the partial transfer if we have previously received some bytes 
        {
            printf("resuming d/l..\n");
            fflush(fp);
            //get the new filesize & sanity check for file; on error quit outer do-loop & return to main
            if (-1==(retval=fstat(fileno(fp), &st)) || !(st_dldata.existing_filesize=st.st_size)) break; 
            //see also: CURLOPT_RANGE for passing a string with our own X-Y range
            curl_easy_setopt(curl, CURLOPT_RESUME_FROM, st.st_size);
            st_dldata.new_bytes_received=0;
        }
        printf("\n\nbytes already received:[%.0f]\n", st_dldata.existing_filesize);

        //re-use the curl handle again & again & again & again... lol
        curl_multi_add_handle(multi_handle, curl);

        do //curl_multi_perform event-loop
        {
            CURLMsg *pMsg;
            int msgs_in_queue;

            while (CURLM_CALL_MULTI_PERFORM == curl_multi_perform(multi_handle, &handle_count));

            //check for any mesages regardless of handle count
            while(pMsg=curl_multi_info_read(multi_handle, &msgs_in_queue))
            {
                long http_response;

                printf("\nmsgs_in_queue:[%d]\n",msgs_in_queue);
                if (CURLMSG_DONE != pMsg->msg)
                {
                    fprintf(stderr,"CURLMSG_DONE != pMsg->msg:[%d]\n", pMsg->msg);
                }
                else
                {
                    printf("pMsg->data.result:[%d] meaning:[%s]\n",pMsg->data.result,curl_easy_strerror(pMsg->data.result));
                    if (CURLE_OK != pMsg->data.result) printf("curl_error_buf:[%s]\n", curl_error_buf);
                    switch(pMsg->data.result)
                    {
                    case CURLE_OK: ///////////////////////////////////////////////////////////////////////////////////////
                    printf("CURLE_OK: ");
                    curl_easy_getinfo(pMsg->easy_handle, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &dl_bytes_remaining);
                    curl_easy_getinfo(pMsg->easy_handle, CURLINFO_SIZE_DOWNLOAD, &dl_bytes_received);
                    if (dl_bytes_remaining == dl_bytes_received)
                    {
                        printf("our work here is done ;)\n");
                        rename(filename, outfilename);
                        retval=1;
                    }
                    else
                    {
                        printf("ouch! st_dldata.new_bytes_received[%f]\n",st_dldata.new_bytes_received);
                        printf("ouch! dl_bytes_received[%f] dl_bytes_remaining[%f]\n",dl_bytes_received,dl_bytes_remaining);
                        retval=dl_bytes_received < dl_bytes_remaining ? 0 : -5;
                    }
                    break; /////////////////////////////////////////////////////////////////////////////////////////////////

                    case CURLE_COULDNT_CONNECT:      //no network connectivity ?
                    case CURLE_OPERATION_TIMEDOUT:   //cos of CURLOPT_LOW_SPEED_TIME
                    case CURLE_COULDNT_RESOLVE_HOST: //host/DNS down ?
                    printf("CURMESSAGE switch handle_count:[%d]\n",handle_count);
                    break; //we'll keep trying

                    default://see: http://curl.haxx.se/libcurl/c/libcurl-errors.html
                    handle_count=0;
                    retval=-5;
                    };


                    //see: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
                    curl_retval=curl_easy_getinfo(pMsg->easy_handle, CURLINFO_RESPONSE_CODE, &http_response);
                    printf("CURLINFO_RESPONSE_CODE HTTP:[%ld]\n", http_response);
                    switch(http_response)
                    {
                    case 0:   //eg connection down  from kick-off ~suggest retrying till some max limit
                    case 200: //yay we at least got to our url
                    case 206: //Partial Content
                    break;

                    case 416:
                    //cannot d/l range ~ either cos no server support
                    //or cos we're asking for an invalid range ~ie: we already d/ld the file
                    printf("HTTP416: either the d/l is already complete or the http server cannot d/l a range\n");
                    retval=2;

                    default: //suggest quitting on an unhandled error
                    handle_count=0;
                    retval=-6;
                    };
                }
            }

            if (handle_count) //select on any active handles
            {
                fd_set fd_read={0}, fd_write={0}, fd_excep={0};
                struct timeval timeout={5,0};
                int select_retval;
                int fd_max;

                curl_multi_fdset(multi_handle, &fd_read, &fd_write, &fd_excep, &fd_max);
                if (-1 == (select_retval=select(fd_max+1, &fd_read, &fd_write, &fd_excep, &timeout)))
                {
                    //errno shall be set to indicate the error
                    fprintf(stderr, "yikes! select error :(\n");
                    handle_count=0;
                    retval=-7;
                    break;
                }
                else{/*check whatever*/}
            }

        } while (handle_count);

        curl_multi_remove_handle(multi_handle,curl);
        printf("continue from here?");
        getchar();        
    }
    while(retval==0);

    curl_multi_cleanup(multi_handle);
    curl_easy_cleanup(curl);
    curl_global_cleanup();
    if (fp) fclose(fp);

    return retval;
}

////////////////////////
int main(void)
{
    int retval;
    printf("\n\ncurl_multi d/l test ~curl version:[%s]\n", curl_version());
    while (1!=(retval=do_dl()))
    {
        printf("retval [%d] continue?\n\n", retval);
        printf("continue?");
        getchar();
    }
    printf("\nend of test!\n\n", retval);
    return retval;
}

/* notes ----

$sudo apt-get install libcurl4-gnutls-dev
$curl-config --libs
-L/usr/lib/i386-linux-gnu -lcurl -Wl,-Bsymbolic-functions

#oook. lets do it:
$gcc -o curltest_multi curltest_multi.c -L/usr/lib/i386-linux-gnu -lcurl -Wl,-Bsymbolic-functions
$./curltest_multi

*/

您可能要记住blah.dl在开始全新测试之前删除该文件。该程序故意不这样做,因此您可以预先截断现有文件以进行测试;)

注意:对于这样的事情,您可能不应该*只是依赖CURLE_COULDNT_CONNECT〜您的代码应该主要是错误处理(如果您的程序严格供个人使用,则可能更少)

编辑

我更新了 curtest_multi.c 来演示easy_handle重用。

注意文档中的以下引用

单次传输完成后,简单句柄仍会添加到多堆栈中。您需要先使用 curl_multi_remove_handle(3) 删除简单句柄,然后使用 curl_easy_cleanup(3) 将其关闭,或者可能为其设置新选项并使用 curl_multi_add_handle(3) 再次添加以开始另一次传输。

于 2012-05-05T01:16:40.947 回答