2

我一直在努力解决一些看起来非常基本的问题，这个问题与使用 Jetty continuations 进行长轮询有关。

为了简单起见，我删除了所有应用程序特定的代码，只留下简单的延续相关代码。

我在下面粘贴我的 servlet 的 doPost 方法。我需要一些专家指导的关键问题是

  • 在下面的代码块中，如果我按原样运行它并触发带有大约 200 字节的 post 正文的 post 请求，那么 500 个长轮询连接的内存量约为 20 MB。
  • 好像我将突出显示的块注释为“减少内存占用:: 下面的注释块”，然后内存占用降至 7 MB

在这两种情况下，我都等待系统稳定，多次调用 GC，然后通过 jConsole 读取内存。它并不精确，但差异是如此之大且可以解释，以至于这里或那里的几个 100 字节的精度无关紧要。

我的问题爆发了，考虑到我的服务器需要保持 100K 连接，如果不是更多的话。在这里，这种无法解释的大小增加最终导致使用了接近 GB 的额外堆。

（是什么导致了这种额外的堆使用，即使从流中读取的内容也没有保留在 doPost 方法的范围之外。但它仍然添加到堆中......我错过了什么？）

   @Override
   protected void doPost(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {

    Continuation cc = ContinuationSupport.getContinuation(req);

    //if continuation is resumed, then send an answer back with 
    //hardcoded answer
    if (cc.isResumed()) {
        String myJson = "{\"1\",\"2\"}";
        res.setContentType("application/json");
        res.setContentLength(myJson.length());
        PrintWriter writer = res.getWriter();
        writer.write(myJson);
        writer.close();
    } 
    // if it is the first call to doPost ( not reentrant call )
    else if (cc.isInitial()) {          

        //START :: decrease memory footprint :: comment this block :: START

        // store the json from the request body in a string
        StringBuffer jsonString = new StringBuffer();
        String line = null;                      
        BufferedReader bufferedReader = req.getReader();
        while ((line = bufferedReader.readLine()) != null) {
            jsonString.append(line);
        }  

        //here jsonString was parsed and some values extracted
        //though that code is removed for the sake of this publish
        // as problem exists irrespective...of any processing

        line = null;            
        bufferedReader.close();
        bufferedReader = null;
        jsonString = null;

        // END :: decrease memory footprint :: comment this block :: END

        cc.setTimeout(150000);        

        cc.suspend();
    }
}
4

1 回答 1

1

是什么导致了这种额外的堆使用...

看看这一行：

BufferedReader bufferedReader = req.getReader();

Note that you are not actually creating a new BufferedReader. When you call getBufferedReader, Jetty creates a BufferedReader which wraps an InputStreamReader which wraps a custom InputStream implementation which wraps a byte buffer. I am pretty sure that by executing the code which reads the entire message, you create large byte buffer inside the request object which stores the entire contents of the message body. Plus the request object maintains a reference to the readers.

At the beginning of the function you called:

Continuation cc = ContinuationSupport.getContinuation(req);

I believe your continuation is holding onto the request which is storing all the data. So the simple act of reading the data is allocating the memory which will be preserved until you discontinue your continuation.

One thing you might try just as an experiment. Change your code to:

BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(req.getInputStream()));

This way Jetty won't allocate it's own readers. Again - I don't know how much data is really stored in the readers compared to the rest of the request object - but it might help a little.

[update]

Another alternative is to avoid the problem. That's what I did (although I was using servlet 3.0 rather than Continuations). I had a resource - let's call it /transfer which would POST some data, then use an AsyncContext to wait for a response. I changed it to two requests with different URLS - /push and /pull. Any time I had some content that needed to be sent from client to server, it would go in the /push request which would then immediately return without creating an AsyncContext. Thus, any storage in the request is freed up right away. Then to wait for the response, I sent a second GET request with no message body. Sure - the request hangs around for a while - but who cares - it does not have any content.

You may have to rethink your problem and determine if you can perform your task in pieces - multiple requests - or whether you really have to handle everything in a single request.

于 2012-12-02T00:30:40.837 回答