ruby - wrk 在 Rack vs Sinatra 基准测试中显示奇怪的结果

Question

我正在使用 sinatra 和 rack 对“hello world”等价物进行基准测试。

有问题的命令wrk -t12 -c400 -d30s：12 个线程，400 个打开的 HTTP 连接，30 秒。

架子：

require 'rack'

app = Proc.new do |env|
    ['200', {'Content-Type' => 'text/html'}, ['A barebones rack app.']]
end

Rack::Handler::Thin.run app

# wrk $ wrk -t12 -c400 -d30s http://localhost:8080
# Running 30s test @ http://localhost:8080
#   12 threads and 400 connections
#   Thread Stats   Avg      Stdev     Max   +/- Stdev
#     Latency    11.82ms   38.97ms 488.51ms   99.32%
#     Req/Sec   705.04    568.62     2.20k    61.82%
#   16576 requests in 30.08s, 1.55MB read
#   Socket errors: connect 157, read 274, write 0, timeout 0
# Requests/sec:    551.05
# Transfer/sec:     52.74KB

西纳特拉：

require 'sinatra'

get '/' do
  status 200
  headers \
    'Content-Type' => 'text/html'
  'A barebones rack app.'
end

# wrk $ wrk -t12 -c400 -d30s http://localhost:4567
# Running 30s test @ http://localhost:4567
#   12 threads and 400 connections
#   Thread Stats   Avg      Stdev     Max   +/- Stdev
#     Latency    40.12ms   90.46ms   1.39s    98.67%
#     Req/Sec   265.47    147.50     1.17k    73.15%
#   90322 requests in 30.08s, 18.78MB read
#   Socket errors: connect 157, read 333, write 0, timeout 0
# Requests/sec:   3002.52
# Transfer/sec:    639.21KB

眼镜：

如果 Rack 和 Sinatra 都运行 Thin，为什么 Sinatra 管理 3002.52~ req/s 而纯 Rack 只管理 551.05 req/s？我错过了什么？

Answer 1

I didn't install wrk, I wasn't in the mood for an install, but I ran both your examples using time:

# run.sh
for i in {1..1000}
do 
  curl localhost:8080/ > /dev/null 2>&1
done

# sinatra, run via `bundle exec ruby sinatra.rb -p 8080`
$ time sh run.sh             
sh run.sh  3.67s user 2.79s system 66% cpu 9.768 total

# rack, run via `bin/rackup config.ru`
$ time sh run.sh
sh run.sh  3.65s user 2.87s system 74% cpu 8.799 total

# sinatra with the puma server, by adding the line `set :server, "puma"`
$ time sh run.sh      
sh run.sh  3.67s user 2.71s system 92% cpu 6.924 total

I got very similar results, so perhaps it's something to do with wrk or your system. I'm running OSX Mavericks, but I found this answer that says to benchmark with time you should really use chrt to avoid contention with other processes. I'm not sure what the OSX equivalent of chrt is though. Perhaps that's what is happening on your system, or perhaps the ports are running at different speeds, for whatever reason.

I also sandboxed my gems using bundle install --binstubs --path vendor.noindex, perhaps that would have an effect on your system too.

Otherwise, I can't see any reason for such disparity. I a little surprised by the results I got as I'd expect Sinatra to be quite a bit heavier than Rack, but perhaps things seem different with more complex or larger apps. Or perhaps not, Sinatra is such a neat little library.