13

我正在寻找一个多线程而不是多进程的python网络服务器（例如mod_python for apache）。我希望它是多线程的，因为我想要一个内存对象缓存，供各种 http 线程使用。我的网络服务器做了很多昂贵的事情并计算了一些需要缓存在内存中以备将来使用以避免重新计算的大型数组。这在多进程 Web 服务器环境中是不可能的。将这些信息存储在 memcache 中也不是一个好主意，因为数组很大，并且将它们存储在 memcache 中会导致来自 memcache 的数据的反序列化，除了 IPC 的额外开销。

我使用 BaseHttpServer 实现了一个简单的网络服务器，它提供了良好的性能，但几个小时后就卡住了。我需要一些更成熟的网络服务器。是否可以将 apache 配置为在线程模型下使用 mod_python 以便我可以进行一些对象缓存？

4

11 回答 11

16

樱桃派。网站上列出的功能：

  • 一个快速的、符合 HTTP/1.1 的、WSGI 线程池网络服务器。通常，CherryPy 本身每页只需要 1-2 毫秒！
  • Support for any other WSGI-enabled webserver or adapter, including Apache, IIS, lighttpd, mod_python, FastCGI, SCGI, and mod_wsgi
  • Easy to run multiple HTTP servers (e.g. on multiple ports) at once
  • A powerful configuration system for developers and deployers alike
  • A flexible plugin system
  • Built-in tools for caching, encoding, sessions, authorization, static content, and many more
  • A native mod_python adapter
  • A complete test suite
  • Swappable and customizable...everything.
  • Built-in profiling, coverage, and testing support.
于 2008-10-17T19:33:25.227 回答
7

Consider reconsidering your design. Maintaining that much state in your webserver is probably a bad idea. Multi-process is a much better way to go for stability.

Is there another way to share state between separate processes? What about a service? Database? Index?

It seems unlikely that maintaining a huge array of data in memory and relying on a single multi-threaded process to serve all your requests is the best design or architecture for your app.

于 2008-10-17T19:35:04.240 回答
6

Twisted can serve as such a web server. While not multithreaded itself, there is a (not yet released) multithreaded WSGI container present in the current trunk. You can check out the SVN repository and then run:

twistd web --wsgi=your.wsgi.application
于 2008-10-18T03:32:42.917 回答
3

Its hard to give a definitive answer without knowing what kind of site you are working on and what kind of load you are expecting. Sub second performance may be a serious requirement or it may not. If you really need to save that last millisecond then you absolutely need to keep your arrays in memory. However as others have suggested it is more than likely that you don't and could get by with something else. Your usage pattern of the data in the array may affect what kinds of choices you make. You probably don't need access to the entire set of data from the array all at once so you could break your data up into smaller chunks and put those chunks in the cache instead of the one big lump. Depending on how often your array data needs to get updated you might make a choice between memcached, local db (berkley, sqlite, small mysql installation, etc) or a remote db. I'd say memcached for fairly frequent updates. A local db for something in the frequency of hourly and remote for the frequency of daily. One thing to consider also is what happens after a cache miss. If 50 clients all of a sudden get a cache miss and all of them at the same time decide to start regenerating those expensive arrays your box(es) will quickly be reduced to 8086's. So you have to take in to consideration how you will handle that. Many articles out there cover how to recover from cache misses. Hope this is helpful.

于 2008-10-17T20:34:06.773 回答
2

不是多线程的，但扭曲的可能会满足您的需求。

于 2008-10-17T19:22:28.320 回答
2

也许您在 Python 中使用BaseHttpServer. 它没有理由“卡住”，并且使用并且实现一个简单的线程服务器BaseHttpServer应该threading不难。

另外，请参阅http://pymotw.com/2/BaseHTTPServer/index.html#module-BaseHTTPServer关于实现一个简单的多线程服务器HTTPServer和ThreadingMixIn

于 2008-10-17T19:27:36.050 回答
2

您可以改为使用可从每个进程访问的分布式缓存，memcached就是您想到的示例。

于 2008-10-17T19:31:15.980 回答
2

web.py has made me happy in the past. Consider checking it out.

But it does sound like an architectural redesign might be the proper, though more expensive, solution.

于 2008-10-18T17:12:17.050 回答
1

我个人和专业都使用 CherryPy，对此我感到非常满意。我什至做了你描述的那种事情，比如拥有全局对象缓存，在后台运行其他线程等等。它与 Apache 集成得很好；只需将 CherryPy 作为绑定到 localhost 的独立服务器运行，然后使用 Apachemod_proxy并mod_rewrite让 Apache 透明地将您的请求转发给 CherryPy。

CherryPy 网站是http://cherrypy.org/

于 2008-10-17T19:30:57.803 回答
1

I actually had the same issue recently. Namely: we wrote a simple server using BaseHTTPServer and found that the fact that it's not multi-threaded was a big drawback.

My solution was to port the server to Pylons (http://pylonshq.com/). The port was fairly easy and one benefit was it's very easy to create a GUI using Pylons so I was able to throw a status page on top of what's basically a daemon process.

I would summarize Pylons this way:

  • it's similar to Ruby on Rails in that it aims to be very easy to deploy web apps
  • it's default templating language, Mako, is very nice to work with
  • it uses a system of routing urls that's very convenient
  • for us performance is not an issue, so I can't guarantee that Pylons would perform adequately for your needs
  • you can use it with Apache & Lighthttpd, though I've not tried this

We also run an app with Twisted and are happy with it. Twisted has good performance, but I find Twisted's single-threaded/defer-to-thread programming model fairly complicated. It has lots of advantages, but would not be my choice for a simple app.

Good luck.

于 2008-10-18T17:04:14.523 回答
0

Just to point out something different from the usual suspects...

Some years ago while I was using Zope 2.x I read about Medusa as it was the web server used for the platform. They advertised it to work well under heavy load and it can provide you with the functionality you asked.

于 2008-10-17T20:31:57.823 回答