0

我有一个 1GB/s 专用、4GB 内存和 4cpus 的专用服务器。我有静态文件可供下载（从 300mb 到 900mb）。我正在测试 Apache、Nginx 和 Lighttpd。

Apache 制造了太多的威胁，在 200 次连接后它变得非常高，所以 apache 是不行的......

Nginx 在 100 次连接后变得非常高，所以它也是不行的。

到目前为止，Lighttpd 和单线程服务器一样非常好。有 500 个并发连接时，负载保持在 0.90 - 1.10（非常好），但我面临下载速度问题，即使我有 1GBps 专用端口，它也会变慢，我看到 iptraf 并且有 500 个并发连接它不会超过250000 KB/秒。使用 apache 和 nginx 有时它会在服务器的上游达到 700000 KB/s。我在配置中的 sendfile 和 writev 之间切换，结果相同。

我没有使用任何 php 或 fast-cgi，只是直接下载到文件，例如：http ://www.myserver.com/file.zip并下载文件。

我将在此处附上一些信息，以帮助我弄清楚。

内核 2.6

lighttpd.conf

# lighttpd configuration file
#
# use it as a base for lighttpd 1.0.0 and above
#
# $Id: lighttpd.conf,v 1.7 2004/11/03 22:26:05 weigon Exp $

############ Options you really have to take care of ####################

## modules to load
# at least mod_access and mod_accesslog should be loaded
# all other module should only be loaded if really neccesary
# - saves some time
# - saves memory
server.modules              = (
#                               "mod_rewrite",
#                               "mod_redirect",
#                               "mod_alias",
                                "mod_access",
#                               "mod_cml",
#                               "mod_trigger_b4_dl",
#                               "mod_auth",
#                               "mod_status",
#                               "mod_setenv",
#                               "mod_proxy_core",
#                               "mod_proxy_backend_http",
#                               "mod_proxy_backend_fastcgi",
#                               "mod_proxy_backend_scgi",
#                               "mod_proxy_backend_ajp13",
#                               "mod_simple_vhost",
#                               "mod_evhost",
#                               "mod_userdir",
#                               "mod_cgi",
#                               "mod_compress",
#                               "mod_ssi",
#                               "mod_usertrack",
#                               "mod_expire",
#                               "mod_secdownload",
#                               "mod_rrdtool",
                                "mod_accesslog" )

## a static document-root, for virtual-hosting take look at the
## server.virtual-* options
server.document-root        = "/usr/share/nginx/html/"

## where to send error-messages to
server.errorlog             = "/www/logs/lighttpd.error.log"

# files to check for if .../ is requested
index-file.names            = ( "index.php", "index.html",
                                "index.htm", "default.htm" )

## set the event-handler (read the performance section in the manual)
# server.event-handler = "freebsd-kqueue" # needed on OS X

server.event-handler = "linux-sysepoll"
#server.network-backend = "linux-sendfile"
server.network-backend = "writev"

# mimetype mapping
mimetype.assign             = (
  ".pdf"          =>      "application/pdf",
  ".sig"          =>      "application/pgp-signature",
  ".spl"          =>      "application/futuresplash",
  ".class"        =>      "application/octet-stream",
  ".ps"           =>      "application/postscript",
  ".torrent"      =>      "application/x-bittorrent",
  ".dvi"          =>      "application/x-dvi",
  ".gz"           =>      "application/x-gzip",
  ".pac"          =>      "application/x-ns-proxy-autoconfig",
  ".swf"          =>      "application/x-shockwave-flash",
  ".tar.gz"       =>      "application/x-tgz",
  ".tgz"          =>      "application/x-tgz",
  ".tar"          =>      "application/x-tar",
  ".zip"          =>      "application/zip",
  ".mp3"          =>      "audio/mpeg",
  ".m3u"          =>      "audio/x-mpegurl",
  ".wma"          =>      "audio/x-ms-wma",
  ".wax"          =>      "audio/x-ms-wax",
  ".ogg"          =>      "application/ogg",
  ".wav"          =>      "audio/x-wav",
  ".gif"          =>      "image/gif",
  ".jpg"          =>      "image/jpeg",
  ".jpeg"         =>      "image/jpeg",
  ".png"          =>      "image/png",
  ".xbm"          =>      "image/x-xbitmap",
  ".xpm"          =>      "image/x-xpixmap",
  ".xwd"          =>      "image/x-xwindowdump",
  ".css"          =>      "text/css",
  ".html"         =>      "text/html",
  ".htm"          =>      "text/html",
  ".js"           =>      "text/javascript",
  ".asc"          =>      "text/plain",
  ".c"            =>      "text/plain",
  ".cpp"          =>      "text/plain",
  ".log"          =>      "text/plain",
  ".conf"         =>      "text/plain",
  ".text"         =>      "text/plain",
  ".txt"          =>      "text/plain",
  ".dtd"          =>      "text/xml",
  ".xml"          =>      "text/xml",
  ".mpeg"         =>      "video/mpeg",
  ".mpg"          =>      "video/mpeg",
  ".mov"          =>      "video/quicktime",
  ".qt"           =>      "video/quicktime",
  ".avi"          =>      "video/x-msvideo",
  ".asf"          =>      "video/x-ms-asf",
  ".asx"          =>      "video/x-ms-asf",
  ".wmv"          =>      "video/x-ms-wmv",
  ".bz2"          =>      "application/x-bzip",
  ".tbz"          =>      "application/x-bzip-compressed-tar",
  ".tar.bz2"      =>      "application/x-bzip-compressed-tar"
 )

# Use the "Content-Type" extended attribute to obtain mime type if possible
#mimetype.use-xattr        = "enable"


## send a different Server: header
## be nice and keep it at lighttpd
# server.tag                 = "lighttpd"

#### accesslog module
accesslog.filename          = "/www/logs/access.log"

## deny access the file-extensions
#
# ~    is for backupfiles from vi, emacs, joe, ...
# .inc is often used for code includes which should in general not be part
#      of the document-root
url.access-deny             = ( "~", ".inc" )

$HTTP["url"] =~ "\.pdf$" {
  server.range-requests = "disable"
}

##
# which extensions should not be handle via static-file transfer
#
# .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

######### Options that are good to be but not neccesary to be changed #######

## bind to port (default: 80)
#server.port                = 81

## bind to localhost (default: all interfaces)
#server.bind                = "grisu.home.kneschke.de"

## error-handler for status 404
#server.error-handler-404   = "/error-handler.html"
#server.error-handler-404   = "/error-handler.php"

## to help the rc.scripts
#server.pid-file            = "/var/run/lighttpd.pid"


###### virtual hosts
##
##  If you want name-based virtual hosting add the next three settings and load
##  mod_simple_vhost
##
## document-root =
##   virtual-server-root + virtual-server-default-host + virtual-server-docroot
## or
##   virtual-server-root + http-host + virtual-server-docroot
##
#simple-vhost.server-root   = "/home/weigon/wwwroot/servers/"
#simple-vhost.default-host  = "grisu.home.kneschke.de"
#simple-vhost.document-root = "/pages/"


##
## Format: <errorfile-prefix><status-code>.html
## -> ..../status-404.html for 'File not found'
#server.errorfile-prefix    = "/home/weigon/projects/lighttpd/doc/status-"

## virtual directory listings
#dir-listing.activate       = "enable"

## enable debugging
#debug.log-request-header     = "enable"
#debug.log-response-header    = "enable"
#debug.log-request-handling   = "enable"
#debug.log-file-not-found     = "enable"
#debug.log-condition-handling = "enable"

### only root can use these options
#
# chroot() to directory (default: no chroot() )
#server.chroot              = "/"

## change uid to <uid> (default: don't care)
#server.username            = "wwwrun"

## change uid to <uid> (default: don't care)
#server.groupname           = "wwwrun"

#### compress module
#compress.cache-dir         = "/tmp/lighttpd/cache/compress/"
#compress.filetype          = ("text/plain", "text/html")

#### proxy module
## read proxy.txt for more info
#$HTTP["url"] =~ "\.php$" {
#   proxy-core.balancer = "round-robin"
#   proxy-core.allow-x-sendfile = "enable"
#   proxy-core.protocol = "http"
#   proxy-core.backends = ( "192.168.0.101:80" )
#   proxy-core.max-pool-size = 16
#}


#### fastcgi module
## read fastcgi.txt for more info
## for PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini
#$HTTP["url"] =~ "\.php$" {
#   proxy-core.balancer = "round-robin"
#   proxy-core.allow-x-sendfile = "enable"
#   proxy-core.check-local = "enable"
#   proxy-core.protocol = "fastcgi"
#   proxy-core.backends = ( "unix:/tmp/php-fastcgi.sock" )
#   proxy-core.max-pool-size = 16
#}


#### CGI module
#cgi.assign                 = ( ".pl"  => "/usr/bin/perl",
#                               ".cgi" => "/usr/bin/perl" )
#

#### SSL engine
#ssl.engine                 = "enable"
#ssl.pemfile                = "server.pem"

#### status module
#status.status-url          = "/server-status"
#status.config-url          = "/server-config"

#### auth module
## read authentication.txt for more info
#auth.backend               = "plain"
#auth.backend.plain.userfile = "lighttpd.user"
#auth.backend.plain.groupfile = "lighttpd.group"

#auth.backend.ldap.hostname = "localhost"
#auth.backend.ldap.base-dn  = "dc=my-domain,dc=com"
#auth.backend.ldap.filter   = "(uid=$)"

#auth.require               = ( "/server-status" =>
#                               (
#                                 "method"  => "digest",
#                                 "realm"   => "download archiv",
#                                 "require" => "user=jan"
#                               ),
#                               "/server-config" =>
#                               (
#                                 "method"  => "digest",
#                                 "realm"   => "download archiv",
#                                 "require" => "valid-user"
#                               )
#                             )

#### url handling modules (rewrite, redirect, access)
#url.rewrite                = ( "^/$"             => "/server-status" )
#url.redirect               = ( "^/wishlist/(.+)" => "http://www.123.org/$1" )
#### both rewrite/redirect support back reference to regex conditional using %n
#$HTTP["host"] =~ "^www\.(.*)" {
#  url.redirect            = ( "^/(.*)" => "http://%1/$1" )
#}

#
# define a pattern for the host url finding
# %% => % sign
# %0 => domain name + tld
# %1 => tld
# %2 => domain name without tld
# %3 => subdomain 1 name
# %4 => subdomain 2 name
#
#evhost.path-pattern        = "/home/storage/dev/www/%3/htdocs/"

#### expire module
#expire.url                 = ( "/buggy/" => "access 2 hours", "/asdhas/" => "access plus 1 seconds 2 minutes")

#### ssi
#ssi.extension              = ( ".shtml" )

#### rrdtool
#rrdtool.binary             = "/usr/bin/rrdtool"
#rrdtool.db-name            = "/var/www/lighttpd.rrd"

#### setenv
#setenv.add-request-header  = ( "TRAV_ENV" => "mysql://user@host/db" )
#setenv.add-response-header = ( "X-Secret-Message" => "42" )

## for mod_trigger_b4_dl
# trigger-before-download.gdbm-filename = "/home/weigon/testbase/trigger.db"
# trigger-before-download.memcache-hosts = ( "127.0.0.1:11211" )
# trigger-before-download.trigger-url = "^/trigger/"
# trigger-before-download.download-url = "^/download/"
# trigger-before-download.deny-url = "http://127.0.0.1/index.html"
# trigger-before-download.trigger-timeout = 10

## for mod_cml
## don't forget to add index.cml to server.indexfiles
# cml.extension               = ".cml"
# cml.memcache-hosts          = ( "127.0.0.1:11211" )

#### variable usage:
## variable name without "." is auto prefixed by "var." and becomes "var.bar"
#bar = 1
#var.mystring = "foo"

## integer add
#bar += 1
## string concat, with integer cast as string, result: "www.foo1.com"
#server.name = "www." + mystring + var.bar + ".com"
## array merge
#index-file.names = (foo + ".php") + index-file.names
#index-file.names += (foo + ".php")

#### include
#include /etc/lighttpd/lighttpd-inc.conf
## same as above if you run: "lighttpd -f /etc/lighttpd/lighttpd.conf"
#include "lighttpd-inc.conf"

#### include_shell
#include_shell "echo var.a=1"
## the above is same as:
#var.a=1

sysctl.conf

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

# These ensure that TIME_WAIT ports either get reused or closed fast.
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_tw_recycle = 1

# TCP memory
net.core.rmem_max = 16777216
net.core.rmem_default = 16777216
net.core.netdev_max_backlog = 262144
net.core.somaxconn = 262144

net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2

# For Large File Hosting Servers
net.core.wmem_max = 1048576
#net.ipv4.tcp_wmem = 4096 87380 524288
net.ipv4.tcp_wmem = 4096 524288 16777216

实际顶部命令

top - 16:15:57 up 6 days, 19:30,  2 users,  load average: 1.05, 0.85, 0.83
Tasks: 143 total,   1 running, 142 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.6%us,  2.8%sy,  0.0%ni, 64.7%id, 30.8%wa,  0.0%hi,  1.1%si,  0.0%st
Mem:   3914664k total,  3729404k used,   185260k free,     1676k buffers
Swap:  8388600k total,     9984k used,  8378616k free,  3340832k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
28590 root      20   0  518m  75m  71m D 13.1  2.0   1:12.24 lighttpd
28660 root      20   0 15016 1104  812 R  1.9  0.0   0:00.02 top
    1 root      20   0 19328  620  396 S  0.0  0.0   0:03.74 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.02 kthreadd
    3 root      RT   0     0    0    0 S  0.0  0.0   0:00.14 migration/0
    4 root      20   0     0    0    0 S  0.0  0.0   0:00.12 ksoftirqd/0
    5 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
    6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    7 root      RT   0     0    0    0 S  0.0  0.0   0:00.32 migration/1
    8 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/1
    9 root      20   0     0    0    0 S  0.0  0.0   0:01.96 ksoftirqd/1
   10 root      RT   0     0    0    0 S  0.0  0.0   0:00.19 watchdog/1
   11 root      RT   0     0    0    0 S  0.0  0.0   0:01.00 migration/2
   12 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/2
   13 root      20   0     0    0    0 S  0.0  0.0   5:04.44 ksoftirqd/2
   14 root      RT   0     0    0    0 S  0.0  0.0   0:00.23 watchdog/2
   15 root      RT   0     0    0    0 S  0.0  0.0   0:00.50 migration/3
   16 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/3
   17 root      20   0     0    0    0 S  0.0  0.0   0:01.84 ksoftirqd/3
   18 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/3

iostat

Linux 2.6.32-220.7.1.el6.x86_64 (zlin)  05/01/2012      _x86_64_        (4 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.57    0.00    3.95   30.76    0.00   64.72

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda             109.58     38551.74       149.33 22695425748   87908220

netstat -an |grep :80 |wc -l

259

iptraf

247270.0 kbits/sec

我应该改变什么来让客户端下载速度更快，他们说有时下载速度低于 10 KB/s

4

2 回答 2

2

因此，看起来磁盘 I/O 是您查看topand的问题iostat。您已经用完了所有磁盘缓存，系统正在等待从磁盘读取数据，然后才能将其发送到 NIC。

我会尝试的第一件事是更改为：

server.network-backend = "linux-sendfile"

因为这将改善缓冲区的使用（有点，不是应该的那么多）。

您可以对缓存典型工作负载需要多少内存进行粗略计算（简单地说，只需将最常用的 100 个文件的大小相加即可）。我猜它会比你拥有的 4GB 内存要多得多，所以接下来要做的就是获得更快的磁盘驱动器或更多的内存。

这就是人们使用内容交付网络 (CDN) 来交付大量带宽（通常以大文件的形式出现）的原因。

于 2012-06-16T20:20:53.067 回答
0

这里的问题不在于您的 Web 服务器，而在于 HTTP 并没有真正设计为文件下载协议。它是超文本传输​​协议，许多围绕 HTTP 的决定都集中在超链接文本方面——文件大小预计会很小，低于几十 Kb，当然低于 1 Mb。Web 基础设施在许多数据缓存等方法中利用了这一事实。我建议不要将 HTTP 用于它真正不是为它设计的东西，而是考虑使用不同的传输机制。

FTP：文件传输协议。FTP 专为传输任意大小的文件而设计，与 HTTP 软件的假设不同。如果您所做的只是静态下载，您的网页 HTML 可以使用 ftp:// 链接链接到静态文件，并且配置 FTP 服务器以允许匿名下载通常很简单。查看您的 FTP 服务器的文档以获取详细信息。自 IE6/FF2 以来的浏览器本身就支持基本的 FTP——普通用户的工作流程与平常没有什么不同。这可能不是最好的方法，因为 FTP 早在 HTTP 之前就设计好了，而且正如 Perry 提到的，早在我们有半个 gig 文件之前。

CDN：使用像 Amazon 的 S3 这样的内容交付网络在技术上并不能绕过 HTTP，但它让您不必担心您的用户会像您看到的那样使您的服务器过载。

BitTorrent：如果您的用户更精通技术，请考虑将您的服务器设置为无限期地播种静态文件，然后在您的网站上发布磁力链接。在最坏的情况下，单个用户将体验从您的服务器直接下载，使用实际上知道如何处理大文件的协议。在最好的情况下，您的数百名用户将互相吸食和播种，从而大大减少您的服务器负载。是的，这要求您的用户知道如何运行和配置 bittorrent，这可能并非如此，但它仍然是一个有趣的文件下载范例。

于 2012-05-05T03:09:19.157 回答