7

Here is a python script that loads a url and captures response time:

import urllib2
import time

opener = urllib2.build_opener()
request = urllib2.Request('http://example.com')

start = time.time()
resp = opener.open(request)
resp.read()
ttlb = time.time() - start

Since my timer is wrapped around the whole request/response (including read()), this will give me the TTLB (time to last byte).

I would also like to get the TTFB (time to first byte), but am not sure where to start/stop my timing. Is urllib2 granular enough for me to add TTFB timers? If so, where would they go?

4

4 回答 4

8

你应该使用pycurl，而不是urllib2

  1. install pyCurl：
    您可以使用 pip / easy_install，或从源代码安装。

    easy_install pyCurl

    也许你应该是一个超级用户。

  2. 用法： 

    import pycurl
import sys 
import json

WEB_SITES = sys.argv[1]

def main():
    c = pycurl.Curl()
    c.setopt(pycurl.URL, WEB_SITES)              #set url
    c.setopt(pycurl.FOLLOWLOCATION, 1)  
    content = c.perform()                        #execute 
    dns_time = c.getinfo(pycurl.NAMELOOKUP_TIME) #DNS time
    conn_time = c.getinfo(pycurl.CONNECT_TIME)   #TCP/IP 3-way handshaking time
    starttransfer_time = c.getinfo(pycurl.STARTTRANSFER_TIME)  #time-to-first-byte time
    total_time = c.getinfo(pycurl.TOTAL_TIME)  #last requst time
    c.close()

    data = json.dumps({'dns_time':dns_time,         
                       'conn_time':conn_time,        
                       'starttransfer_time':starttransfer_time,    
                       'total_time':total_time})
    return data

if __name__ == "__main__":    
    print main()
于 2016-08-12T10:21:47.343 回答
2

Using your current open / read pair there's only one other timing point possible - between the two.

The open() call should be responsible for actually sending the HTTP request, and should (AFAIK) return as soon as that has been sent, ready for your application to actually read the response via read().

Technically it's probably the case that a long server response would make your application block on the call to read(), in which case this isn't TTFB.

However if the amount of data is small then there won't be much difference between TTFB and TTLB anyway. For a large amount of data, just measure how long it takes for read() to return the first smallest possible chunk.

于 2009-04-13T16:51:11.267 回答
1

要获得良好的接近度，您必须阅读（1）。并打乱时间。

它对我来说效果很好。您应该记住的唯一一件事是：python 可能会在调用 read(1) 时加载超过一个字节。取决于它的内部缓冲区。但我认为大多数工具的行为都会不准确。

import urllib2
import time

opener = urllib2.build_opener()
request = urllib2.Request('http://example.com')

start = time.time()
resp = opener.open(request)
# read one byte
resp.read(1)
ttfb = time.time() - start
# read the rest
resp.read()
ttlb = time.time() - start
于 2012-06-14T10:12:53.310 回答
1

默认情况下，在 urllib2 中的 HTTP 打开的实现在读取时没有回调。HTTP 协议的 OOTB 打开器是urllib2.HTTPHandler，它用于httplib.HTTPResponse通过套接字进行实际读取。

理论上，您可以编写自己的 HTTPResponse 和 HTTPHandler 子类，并使用install_opener将其作为默认开启程序安装到 urllib2 中。这将是不平凡的，但不是非常痛苦，所以如果您基本上从标准库复制并粘贴当前的 HTTPResponse 实现并调整其中的begin()方法以在从套接字开始读取时执行一些处理或回调。

于 2009-04-13T17:27:51.693 回答