4

我将 Django 1.5.3 与 gunicorn 18.0 和 lighttpd 一起使用。我使用 lighttpd 提供静态和动态内容：

$HTTP["host"] == "www.mydomain.com" {
    $HTTP["url"] !~ "^/media/|^/static/|^/apple-touch-icon(.*)$|^/favicon(.*)$|^/robots\.txt$" {
            proxy.balance = "hash"
            proxy.server = ( "" => ("myserver" =>
                       ( "host" => "127.0.0.1", "port" => 8013 )
            ))
    }

    $HTTP["url"] =~ "^/media|^/static|^/apple-touch-icon(.*)$|^/favicon(.*)$|^/robots\.txt$" {
            alias.url = (
                    "/media/admin/" => "/var/www/virtualenvs/mydomain/lib/python2.7/site-packages/django/contrib/admin/static/admin/",
                    "/media" => "/var/www/mydomain/mydomain/media",
                    "/static" => "/var/www/mydomain/mydomain/static"
            )
    }

    url.rewrite-once = (
            "^/apple-touch-icon(.*)$" => "/media/img/apple-touch-icon$1",
            "^/favicon(.*)$" => "/media/img/favicon$1",
            "^/robots\.txt$" => "/media/robots.txt"
    )
}

我已经尝试过以许多不同的方式运行 gunicorn（通过 supervisord），但我无法对其进行更好的优化，而不是它可以处理大约 1100 个并发连接。在我的项目中，我需要大约 10000-15000 个连接

command = /var/www/virtualenvs/myproject/bin/python /var/www/myproject/manage.py run_gunicorn -b 127.0.0.1:8013 -w 9 -k gevent --preload --settings=myproject.settings
command = /var/www/virtualenvs/myproject/bin/python /var/www/myproject/manage.py run_gunicorn -b 127.0.0.1:8013 -w 10 -k eventlet --worker_connections=1000 --settings=myproject.settings --max-requests=10000
command = /var/www/virtualenvs/myproject/bin/python /var/www/myproject/manage.py run_gunicorn -b 127.0.0.1:8013 -w 20 -k gevent --settings=myproject.settings --max-requests=1000
command = /var/www/virtualenvs/myproject/bin/python /var/www/myproject/manage.py run_gunicorn -b 127.0.0.1:8013 -w 40 --settings=myproject.settings

在同一台服务器上还有大约 10 个其他项目，但 CPU 和 RAM 都很好，所以这应该不是问题，对吧？

我进行了负载测试，结果如下： 负载测试

在大约 1100 个连接处，我的 lighttpd 错误日志说类似的内容，这就是负载测试显示连接丢失的地方：

2013-10-31 14:06:51: (mod_proxy.c.853) write failed: Connection timed out 110
2013-10-31 14:06:51: (mod_proxy.c.939) proxy-server disabled: 127.0.0.1 8013 83
2013-10-31 14:06:51: (mod_proxy.c.1316) no proxy-handler found for: /

... after about one minute
2013-10-31 14:07:02: (mod_proxy.c.1361) proxy - re-enabled: 127.0.0.1 8013

这些东西也时不时出现：

2013-10-31 14:06:55: (network_linux_sendfile.c.94) writev failed: Connection timed out 600
2013-10-31 14:06:55: (mod_proxy.c.853) write failed: Connection timed out 110
...
2013-10-31 14:06:57: (mod_proxy.c.828) establishing connection failed: Connection timed out
2013-10-31 14:06:57: (mod_proxy.c.939) proxy-server disabled: 127.0.0.1 8013 45

那么如何调整 gunicorn/lighttpd 以更快地提供更多连接？我可以优化什么？你知道任何其他/更好的设置吗？

非常感谢您的帮助！

更新：更多服务器信息

root@django ~ # 顶部

top - 15:28:38 up 100 days,  9:56,  1 user,  load average: 0.11, 0.37, 0.76
Tasks: 352 total,   1 running, 351 sleeping,   0 stopped,   0 zombie
Cpu(s): 33.0%us,  1.6%sy,  0.0%ni, 64.2%id,  0.4%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:  32926156k total, 17815984k used, 15110172k free,   342096k buffers
Swap: 23067560k total,        0k used, 23067560k free,  4868036k cached

root@django ~ # iostat

Linux 2.6.32-5-amd64 (django.myserver.com)   10/31/2013      _x86_64_        (4 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
      33.00    0.00    2.36    0.40    0.00   64.24

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sda             137.76       980.27      2109.21  119567783  257268738
sdb              24.23       983.53      2112.25  119965731  257639874
sdc              24.25       985.79      2110.14  120241256  257382998
md0               0.00         0.00         0.00        400          0
md1               0.00         0.00         0.00        284          6
md2            1051.93        38.93      4203.96    4748629  512773952

root@django ~ # netstat -an |grep :80 |wc -l

7129

内核设置：

echo "10152 65535" > /proc/sys/net/ipv4/ip_local_port_range
sysctl -w fs.file-max=128000
sysctl -w net.ipv4.tcp_keepalive_time=300
sysctl -w net.core.somaxconn=250000
sysctl -w net.ipv4.tcp_max_syn_backlog=2500
sysctl -w net.core.netdev_max_backlog=2500
ulimit -n 10240
4

0 回答 0