2

系统在一段时间后崩溃,但有以下异常:

Python Pycassa 异常读取:

错误:尝试连接到每台服务器两次,但均未成功。最后一次失败是 TTransportException: Could not connect to 10.242.253.141:9160

Cassandra 系统日志

 WARN [MemoryMeter:1] 2012-07-31 12:44:17,555 Memtable.java (line 197) setting live ratio to minimum of 1.0 instead of 0.16881217597040166
 INFO [MemoryMeter:1] 2012-07-31 12:44:17,555 Memtable.java (line 213) CFS(Keyspace='system', ColumnFamily='HintsColumnFamily') liveRatio is 1.0 (just-counted was 1.0).  calculation took 28ms for 122 columns
 WARN [MemoryMeter:1] 2012-07-31 12:44:22,583 Memtable.java (line 197) setting live ratio to minimum of 1.0 instead of 0.11368422493368459
 INFO [MemoryMeter:1] 2012-07-31 12:44:22,586 Memtable.java (line 213) CFS(Keyspace='system', ColumnFamily='HintsColumnFamily') liveRatio is 1.0 (just-counted was 1.0).  calculation took 39ms for 358 columns
 INFO [GossipTasks:1] 2012-07-31 12:44:22,609 Gossiper.java (line 811) InetAddress /10.204.81.77 is now dead.
 WARN [MemoryMeter:1] 2012-07-31 12:44:25,244 Memtable.java (line 197) setting live ratio to minimum of 1.0 instead of 0.13325571841646536
 INFO [MemoryMeter:1] 2012-07-31 12:44:25,244 Memtable.java (line 213) CFS(Keyspace='system', ColumnFamily='HintsColumnFamily') liveRatio is 1.0 (just-counted was 1.0).  calculation took 46ms for 484 columns
 INFO [StorageServiceShutdownHook] 2012-07-31 12:44:26,679 CassandraDaemon.java (line 218) Stop listening to thrift clients
 INFO [StorageServiceShutdownHook] 2012-07-31 12:44:26,762 MessagingService.java (line 539) Waiting for messaging service to quiesce
 INFO [ACCEPT-/10.242.253.141] 2012-07-31 12:44:26,780 MessagingService.java (line 695) MessagingService shutting down server thread.
ERROR [Thrift:40765] 2012-07-31 12:44:26,858 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down
    at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
    at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
    at org.apache.cassandra.service.StorageProxy.scheduleLocalHint(StorageProxy.java:386)
    at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:339)
    at org.apache.cassandra.service.StorageProxy$2.apply(StorageProxy.java:119)
    at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:260)
    at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:193)
    at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:637)
    at org.apache.cassandra.thrift.CassandraServer.internal_insert(CassandraServer.java:511)
    at org.apache.cassandra.thrift.CassandraServer.insert(CassandraServer.java:519)
    at org.apache.cassandra.thrift.Cassandra$Processor$insert.getResult(Cassandra.java:3016)
    at org.apache.cassandra.thrift.Cassandra$Processor$insert.getResult(Cassandra.java:3004)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
    at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
ERROR [Thrift:42739] 2012-07-31 12:44:26,935 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down
    at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
    at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
    at org.apache.cassandra.service.StorageProxy.scheduleLocalHint(StorageProxy.java:386)
    at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:339)
    at org.apache.cassandra.service.StorageProxy$2.apply(StorageProxy.java:119)
    at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:260)
    at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:193)
    at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:637)
    at org.apache.cassandra.thrift.CassandraServer.internal_batch_mutate(CassandraServer.java:587)
    at org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:595)
    at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3112)
    at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3100)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
    at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

cassanda.yaml

cluster_name: "Test Cluster"

initial_token:

hinted_handoff_enabled: true
max_hint_window_in_ms: 3600000 # one hour
hinted_handoff_throttle_delay_in_ms: 1

authenticator: org.apache.cassandra.auth.AllowAllAuthenticator

authority: org.apache.cassandra.auth.AllowAllAuthority

partitioner: org.apache.cassandra.dht.RandomPartitioner

data_file_directories:
    - /mnt
    - /ebs

commitlog_directory: /mnt/cassandra_log/

key_cache_size_in_mb:

key_cache_save_period: 14400


row_cache_size_in_mb: 0

row_cache_save_period: 0


row_cache_provider: SerializingCacheProvider

saved_caches_directory: /var/lib/cassandra/saved_caches

commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000

seed_provider:
    # Addresses of hosts that are deemed contact points. 
    # Cassandra nodes use this list of hosts to find each other and learn
    # the topology of the ring.  You must change this if you are running
    # multiple nodes!
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # seeds is actually a comma-delimited list of addresses.
          # Ex: "<ip1>,<ip2>,<ip3>"
          - seeds: "127.0.0.1"

flush_largest_memtables_at: 0.45

reduce_cache_sizes_at: 0.45
reduce_cache_capacity_to: 0.45

concurrent_reads: 8
concurrent_writes: 8

memtable_total_space_in_mb: 64

commitlog_total_space_in_mb: 64

memtable_flush_writers: 1

memtable_flush_queue_size: 1

trickle_fsync: false
trickle_fsync_interval_in_kb: 10240

storage_port: 7776

ssl_storage_port: 7001

listen_address: 10.242.253.141


rpc_address: 0.0.0.0
rpc_port: 9160

rpc_keepalive: true

rpc_server_type: sync



thrift_framed_transport_size_in_mb: 15

thrift_max_message_length_in_mb: 16

incremental_backups: false

snapshot_before_compaction: false

auto_snapshot: true

column_index_size_in_kb: 64

in_memory_compaction_limit_in_mb: 64

multithreaded_compaction: false

compaction_throughput_mb_per_sec: 16

compaction_preheat_key_cache: true

rpc_timeout_in_ms: 10000

endpoint_snitch: SimpleSnitch

dynamic_snitch_update_interval_in_ms: 100 
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1

request_scheduler: org.apache.cassandra.scheduler.NoScheduler

index_interval: 128

encryption_options:
    internode_encryption: none
    keystore: conf/.keystore
    keystore_password: cassandra
    truststore: conf/.truststore
    truststore_password: cassandra
    # More advanced defaults below:
    # protocol: TLS
    # algorithm: SunX509
    # store_type: JKS
    # cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA]

那么怎么了?请指导?

4

2 回答 2

4

看起来 Cassandra 进程正在被关闭,大概是因为收到了一个信号。这些行表明:

INFO [StorageServiceShutdownHook] 2012-07-31 12:44:26,679 CassandraDaemon.java (line 218) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-07-31 12:44:26,762 MessagingService.java (line 539) Waiting for messaging service to quiesce
INFO [ACCEPT-/10.242.253.141] 2012-07-31 12:44:26,780 MessagingService.java (line 695) MessagingService shutting down server thread.
ERROR [Thrift:40765] 2012-07-31 12:44:26,858 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down
于 2012-08-01T22:30:08.623 回答
3

我有同样的问题。我想出了两件事首先去

sudo vim /etc/cassandra/cassandra.yaml

并将监听 ip 更改为私有 ip 是您在 AWS 上并将 rpc_ip 更改为 0.0.0.0

还要确保你的服务器至少有 2GB 内存,否则 cassandra 会一直崩溃!!

要在 ubuntu 上启动 cassandra,请尝试以下命令:

sudo service cassandra restart

并检查状态

sudo service cassandra status

希望这可以帮助

于 2014-01-23T04:10:06.600 回答