11

使用If-Modified-Since: Wed, 15 Feb 2012 07:25:00 CETset 直接向后端发送 GET 请求时,Apache 正确返回没有内容的 304。

当我通过 Varnish 3.0.2 发送相同的请求时,它会以 200 响应并重新发送所有内容,即使客户端已经拥有它。显然,这不是对带宽的良好利用。我的理解是 Varnish 支持对这个标头进行智能处理并且应该发送 304,所以我认为我的 .vcl 文件有问题。

Varnishlog 给出了这个:

 16 SessionOpen  c 84.97.17.233 64416 :80
   16 ReqStart     c 84.97.17.233 64416 1597323690
   16 RxRequest    c GET
   16 RxURL        c /fr/CS/CS_AU-Maboreke-6-6-2004.pdf
   16 RxProtocol   c HTTP/1.0
   16 RxHeader     c Host: www.quotaproject.org
   16 RxHeader     c User-Agent: Sprawk/1.3 (http://www.sprawk.com/)
   16 RxHeader     c Accept: */*
   16 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
   16 RxHeader     c Connection: close
   16 RxHeader     c If-Modified-Since: Wed, 15 Feb 2012 07:25:00 CET
   16 VCL_call     c recv lookup
   16 VCL_call     c hash
   16 Hash         c /fr/CS/CS_AU-Maboreke-6-6-2004.pdf
   16 Hash         c www.quotaproject.org
   16 VCL_return   c hash
   16 Hit          c 1597322756
   16 VCL_call     c hit
   16 VCL_acl      c NO_MATCH CTRLF5
   16 VCL_return   c deliver
   16 VCL_call     c deliver deliver
   16 TxProtocol   c HTTP/1.1
   16 TxStatus     c 200
   16 TxResponse   c OK
   16 TxHeader     c Server: Apache
   16 TxHeader     c Last-Modified: Wed, 09 Jun 2004 16:07:50 GMT
   16 TxHeader     c Vary: Accept-Encoding
   16 TxHeader     c Content-Type: application/pdf
   16 TxHeader     c Date: Wed, 22 Feb 2012 18:25:05 GMT
   16 TxHeader     c Age: 12432
   16 TxHeader     c Connection: close
   16 Gzip         c U D - 107685 115763 80 796748 861415
   16 Length       c 98304
   16 ReqEnd       c 1597323690 1329935105.713264704 1329935106.208528996 0.000071526 0.000068426 0.495195866
   16 SessionClose c EOF mode
   16 StatSess     c 84.97.17.233 64416 0 1 1 0 0 0 203 98304

如果我理解正确,该对象已经在 Varnish 的缓存中,因此它不需要联系后端,但它已经知道Last-Modified为什么它不会以 304 响应?

这是我的 VCL 文件:

 backend idea {
  # .host = "www.idea.int";
  .host = "83.145.60.235"; # IDEA's public website IP
  .port = "80";
}
backend qp {
  # .host = "www.quotaproject.org";
  .host = "83.145.60.235"; # IDEA's public website IP
  .port = "80";
}
#
#Below is a commented-out copy of the default VCL logic.  If you
#redefine any of these subroutines, the built-in logic will be
#appended to your code.
#
sub vcl_recv {
  # force domain so that Apache handles the VH correctly
  if (req.http.host ~ "^qp" || req.http.host ~ "quotaproject.org$") {
    set req.http.Host = "www.quotaproject.org";
    set req.backend = qp;
  } else {
    # default to idea.int
     set req.http.Host = "www.idea.int";
     set req.backend = idea;
  }
  # Before anything else we need to fix gzip compression 
  if (req.http.Accept-Encoding) {
      if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
          # No point in compressing these
          remove req.http.Accept-Encoding;
      } else if (req.http.Accept-Encoding ~ "gzip") {
          set req.http.Accept-Encoding = "gzip";
      } else if (req.http.Accept-Encoding ~ "deflate") {
          set req.http.Accept-Encoding = "deflate";
      } else {
          # unknown algorithm
          remove req.http.Accept-Encoding;
      }
  }
  # ajax requests bypass cache. TODO: Make sure you Javascript implementation for AJAX actually sets XMLHttpRequest
  if (req.http.X-Requested-With == "XMLHttpRequest") {
        return(pass);
   }
  if (req.request != "GET" &&
     req.request != "HEAD" &&
     req.request != "PUT" &&
     req.request != "POST" &&
     req.request != "TRACE" &&
     req.request != "OPTIONS" &&
     req.request != "DELETE") {
     /* Non-RFC2616 or CONNECT which is weird. */
     return (pipe);
   }
   # Purge everything url - this isn't the squid way, but works
    if (req.url ~ "^/varnishpurge") {
       if (!client.ip ~ purge) {
            error 405 "Not allowed.";
       }
       if (req.url == "/varnishpurge") {
            ban("req.http.host == " + req.http.host + " && req.url ~ ^/");
            error 841 "Purged site.";
       }
       else {
            ban("req.http.host == " + req.http.host + " && req.url ~ ^" + regsub( req.url, "^/varnishpurge(.*)$", "\1" ) + "$");
            error 842 "Purged page.";
       }
    }
  # spoof the client IP (taken from http://utvbloggen.se/snabb-guide-till-varnish/)
  remove req.http.X-Forwarded-For;
  set req.http.X-Forwarded-For = client.ip;
  # Force delivery from cache even if other things indicate otherwise
  if (req.url ~ "\.(flv)") {
    # pipe flash start away
    return(pipe);
  }
  if (req.url ~ "\.(jpg|jpeg|gif|png|tiff|tif|svg|swf|ico|css|vsd|doc|ppt|pps|xls|pdf|mp3|mp4|m4a|ogg|mov|avi|wmv|sxw|zip|gz|bz2|tgz|tar|rar|odc|odb|odf|odg|odi|odp|ods|odt|sxc|sxd|sxi|sxw|dmg|torrent|deb|msi|iso|rpm)$") {
    # cookies are irrelevant here
    unset req.http.Cookie;
    unset req.http.Authorization; 
  }
  # Force short-circuit to the real site for these dynamic pages
  if (req.url ~ "/customcf/" || req.url ~ "/uid/editData.cfm" || req.url ~ "^/private/") {
    return(pass);
  }
  # Remove user agent, since Apache will server these resources the same way
  if (req.http.User-Agent) {
    set req.http.User-Agent = "";
  }
  if (req.http.Cookie) {
    # removes all cookies named __utm? (utma, utmb...) - tracking thing
    set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__utm.=[^;]+;? *", "\1"); 
    # remove cStates for RHM boxes (the server doesn't need to know these, JS will handle this client-side)
    set req.http.cookie = regsub(req.http.cookie, "(; )?cStates=[^;]*", ""); #cStates might sometimes have a blank value
    # remove ColdFusion session cookie stuff
    if (!req.url ~ "^/publications/" && !req.url ~ "^/uid/admin/") {
      set req.http.cookie = regsub(req.http.cookie, "(; )?CFID=[^;]+", "");
      set req.http.cookie = regsub(req.http.cookie, "(; )?CFTOKEN=[^;]+", "");
    }
    # Remove the cookie header if it's empty after cleanup
    if (req.http.cookie ~ "^;? *$") {
      # The only cookie data left is a semicolon or spaces
      remove req.http.cookie;
    }
  }
}
#
# Called when the requested object was not found in the cache
#
sub vcl_hit {
  # Allow administrators to easily flush the cache from their browser
  if (client.ip ~ CTRLF5) {
    if (req.http.pragma ~ "no-cache" || req.http.Cache-Control ~ "no-cache") {
      set obj.ttl = 0s;
      return(pass);
    }
  }
}
#
# Called when the requested object has been retrieved from the
# backend, or the request to the backend has failed
#
sub vcl_fetch {
  set beresp.grace = 1h;
  # strip the cookie before the image is inserted into cache.
  if (req.url ~ "\.(jpg|jpeg|gif|png|tiff|tif|svg|swf|ico|css|vsd|doc|ppt|pps|xls|pdf|mp3|mp4|m4a|ogg|mov|avi|wmv|sxw|zip|gz|bz2|tgz|tar|rar|odc|odb|odf|odg|odi|odp|ods|odt|sxc|sxd|sxi|sxw|dmg|torrent|deb|msi|iso|rpm)$") {
    remove beresp.http.set-cookie;
    set beresp.ttl = 100w;
  }
  # Remove CF session cookies for everything but the publications subsite
  if (!req.url ~ "^/publications/" && !req.url ~ "/customcf/" && !req.url ~ "^/uid/admin/" && !req.url ~ "^/uid/editData.cfm") {
    remove beresp.http.set-cookie;
  }
  if (beresp.ttl < 48h) {
    set beresp.ttl = 48h;
  }
}
#
# Called before a cached object is delivered to the client
#
sub vcl_deliver {
  # We'll be hiding some headers added by Varnish. We want to make sure people are not seeing we're using Varnish.
  remove resp.http.X-Varnish;
  remove resp.http.Via;
  # We'd like to hide the X-Powered-By headers. Nobody has to know we can run PHP and have version xyz of it.
  remove resp.http.X-Powered-By;
}

任何人都可以看到问题或问题吗?

更新:根据http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3

Note: When handling an If-Modified-Since header field, some
      servers will use an exact date comparison function, rather than a
      less-than function, for deciding whether to send a 304 (Not
      Modified) response.

看来这可能是 Varnish 的行为。我正在发送另一个日期,该日期早于实际文件的最后修改日期,但不完全是 Varnish 中缓存的日期。

4

2 回答 2

8

问题是 If-Modified-Since 请求标头中的非 GMT 时区:

If-Modified-Since: Wed, 15 Feb 2012 07:25:00 CET

根据http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.3

所有 HTTP 日期/时间戳必须以格林威治标准时间 (GMT) 表示,无一例外。

Varnish 将此作为一项严格的要求来实现,而 Apache 更稳健地处理非标准日期格式。这就是您在直接查询 Apache 时观察到不同行为的原因。

于 2014-01-17T18:28:55.050 回答
2

由于这个问题仍然是开放的,没有答案和几个赞成票,我会发布一个答案。

这似乎不是 Varnish 3.0.0(我们正在使用)或您在您的站点上运行的当前版本的 Varnish 的问题。

请求带有过期 If-Modified-Since 标头的内容时的 200 OK 响应:

# curl -z "Wed, 09 Jun 2010 16:07:50 GMT" --head "www.quotaproject.org/robots.txt"
HTTP/1.1 200 OK
Server: Apache
Last-Modified: Tue, 22 Jan 2013 13:23:41 GMT
Vary: Accept-Encoding
Cache-Control: public
Content-Type: text/plain; charset=UTF-8
Date: Mon, 25 Nov 2013 15:00:45 GMT
Age: 69236
Connection: keep-alive
X-Cache: HIT

If-Modified-Since 在 Last-Modified 日期之后的 304 响应:

# curl -z "Wed, 09 Jun 2013 16:07:50 GMT" --head "www.quotaproject.org/robots.txt"
HTTP/1.1 304 Not Modified
Server: Apache
Last-Modified: Tue, 22 Jan 2013 13:23:41 GMT
Vary: Accept-Encoding
Cache-Control: public
Content-Type: text/plain; charset=UTF-8
Date: Mon, 25 Nov 2013 15:00:52 GMT
Age: 69243
Connection: keep-alive
X-Cache: HIT

与您在 varnishlog 输出中给出的示例相同:

# curl -z "Wed, 15 Feb 2012 07:25:00 CET" --head "www.quotaproject.org/fr/CS/CS_AU-Maboreke-6-6-2004.pdf"
HTTP/1.1 304 Not Modified
Server: Apache
Last-Modified: Wed, 09 Jun 2004 16:07:50 GMT
Cache-Control: public
Content-Type: application/pdf
Accept-Ranges: bytes
Date: Mon, 25 Nov 2013 15:08:48 GMT
Age: 335802
Connection: keep-alive
X-Cache: HIT

我会说清漆按预期工作。也许这是您正在使用的 Varnish 版本的问题,或者测试方法有问题。我也看不出你的 VCL 有任何问题。

于 2013-11-25T15:11:27.653 回答