3

我正在 6 节点 Cassandra 集群上测试 Clojure 中 Alia 的性能。即使是多线程，我也只能获得大约 400 次写入/秒。使用 Firebrand Cassandra 驱动程序并在 Java 中手动处理线程，我们能够以 96 个线程获得 5000 次写入/秒。

我在这里使用代理时做错了什么吗？运行它的机器上的 CPU 使用率只有约 25%，看起来真的很低。

更新：根据 Alia 的建议，使用准备好的语句而不是原始语句以同步、单线程方式实现了高达 2500/秒的增益。我仍然需要通过 Clojure 的多线程和单独使用 Alia/底层 Java 驱动程序中内置的异步函数来测试这一点，以查看哪个更快。

更新 2：通过另外利用驱动程序中内置的异步功能，我现在看到了与下面的 mpenet 类似的结果。

(ns alia-perf-test.core
  (:gen-class)
  (:require [qbits.alia :as alia]
            [qbits.hayt :as hayt]))

(defn exec-query [session query]
  (alia/execute session (hayt/->raw query)))

(defmacro time-query
  [expr]
  `(let [start# (. System (nanoTime))
         ret# ~expr]
     (/ (double (- (. System (nanoTime)) start#)) 1000000.0)))

(defn write-entity
  [total-time session entity]
  (let [query (hayt/->raw (hayt/insert :entities (hayt/values entity) (hayt/using :timestamp 1234)))
        query-time (time-query (alia/execute session query))]
      (+ total-time query-time)))

(defn generate-entity []
  {:id (str (java.util.UUID/randomUUID)) :num 0})

(defn write-something
  [write-agent session]
  (send-off write-agent write-entity
        session 
        (generate-entity)))

(defn -main [& args]
  (let [cluster (alia/cluster ["server1"
                               "server2" 
                               "server3" 
                               "server4" 
                               "server5" 
                               "server6"]
                              :pooling-options {:core-connections-per-host [:local 16 :remote 16]
                                                :max-connections-per-host  [:local 1000 :remote 1000]
                                                :max-simultaneous-requests-per-connection [:local 32 :remote 32]
                                                :min-simultaneous-requests-per-connection [:local 16 :remote 16]})
        session (alia/connect cluster)]
    (alia/set-consistency! :any)
    (exec-query session (hayt/create-keyspace :aliaperftest
                                              (hayt/with {:replication
                                                          {:class "NetworkTopologyStrategy"
                                                           :dc1 3 :dc2 3}})))
    (exec-query session (hayt/use-keyspace :aliaperftest))
    (exec-query session (hayt/create-table :entities
                                           (hayt/column-definitions {:id :varchar
                                                                     :num :int
                                                                     :primary-key [:id]})))
    (let [num-entities 10000
          write-agent (agent 0)]
      (dotimes [n num-entities]
        (write-something write-agent session))
      (await write-agent)
      (println "Wrote" num-entities "entities in" @write-agent "ms -"
           (* (/ num-entities @write-agent) 1000.0) "ops/sec"))

    (exec-query session (hayt/drop-table :entities))
    (exec-query session (hayt/drop-keyspace :aliaperftest))
    (alia/shutdown session)
    (alia/shutdown cluster)
    (shutdown-agents)))
4

1 回答 1

0

更新：我已经让它在大约 8 秒内以异步模式（使用 alia 的成功处理程序，具有单个节点）执行 100k 请求，使用原子接收结果并等待所有响应到达。

根据您的设置和批处理，也可以通过使用自定义执行器来从中挤出更多性能，我没有走那么远。见：https ://github.com/mpenet/alia/blob/master/docs/guide.md#executors

于 2013-03-01T22:04:06.867 回答