2

使用 twistedmatrix 时如何压缩和处理响应部分ProxyClient？

我需要检查文本或 javascript 和 ajax 查询/答案。我应该使用handleResponseEnd吗？

我认为它在 handleResponsePart 内部，但看起来我误解了一点或什么，这是我的骨架代码：

from twisted.python import log
from twisted.web import http, proxy

class ProxyClient(proxy.ProxyClient):
    """Mange returned header, content here.

    Use `self.father` methods to modify request directly.
    """
    def handleHeader(self, key, value):
            # change response header here
            log.msg("Header: %s: %s" % (key, value))
            proxy.ProxyClient.handleHeader(self, key, value)

    def handleResponsePart(self, buffer):
    # this part below do not work, 
    # looks like @ this moment i do not have 'Content-Encoding' or 'Content-Type'
    # what am i misunderstading?
            cEncoding = self.father.getAllHeaders().get('Content-Encoding', '')
            cType = self.father.getAllHeaders().get('Content-Type', '')
            print >> sys.stderr, 'Content-Encoding', cEncoding
            print >> sys.stderr, 'Content-Type', cType
            if ('text' in cType.lower() or 'javascript' in cType.lower()) and 'gzip' in cEncoding.lower():

                buf = StringIO(buffer)
                s = gzip.GzipFile(mode="rb", fileobj=buf)
                content = s.read(len(buffer))

                # here process content as it should be gunziped

    proxy.ProxyClient.handleResponsePart(self, buffer)

class ProxyClientFactory(proxy.ProxyClientFactory):
    protocol = ProxyClient

class ProxyRequest(proxy.ProxyRequest):
    protocols = dict(http=ProxyClientFactory)

class Proxy(proxy.Proxy):
    requestFactory = ProxyRequest

class ProxyFactory(http.HTTPFactory):
    protocol = Proxy

从我的日志中我有：

2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Date: Tue, 11 Jun 2013 12:07:25 GMT
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Server: Apache
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Content-Type: text/html;charset=ISO-8859-1
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Content-Encoding: gzip
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Connection: close

因此我应该满足这两个条件！请问我错过了什么？

即使我对第二种方式不感兴趣，也就是删除对请求的接受，像这样，是否可以这样做：（顺便说一句，它看起来不起作用或者测试的网络服务器不关心事实上我们不想接收 gzip 格式的内容）

class ProxyRequest(proxy.ProxyRequest):
    protocols = dict(http=ProxyClientFactory)

    def process(self):
        # removing the accept so that we do not tell "i'm ok with gzip encoded content" and should receive only not gzip-ed
        self.requestHeaders.removeHeader('accept')
        self.requestHeaders.removeHeader('accept-encoding')
4

1 回答 1

2

您必须将数据块收集到中的 StringIO 缓冲区中handleResponsePart，然后使用中的 GzipFile 进行解码handleResponseEnd。

于 2013-06-12T05:59:06.127 回答