1

我正在尝试将 100 万个 LUBM 数据集加载到 Cassandra 中。但是,在大约 200,000 次插入后,我不断收到此异常。

线程“主”com.datastax.driver.core.exceptions.NoHostAvailableException 中的异常:所有主机尝试查询失败(尝试:localhost/127.0.0.1(com.datastax.driver.core.exceptions.DriverException:超时期间读))

服务器还活着。我仍然可以使用 Datastax 开发中心执行 CQL 查询。在此之前,我收到了 WriteTimeoutException,因此我在 .yaml 中调整了一些选项

以下是我系统的规格:

CPU- Intel I5 - 2 核| 内存- 4GB | 硬盘 - 500 GB 4 个驱动器。

我有一个 OWL 文件,从中读取数据并将其插入 Cassandra。我有混合的工作量。有时,根据条件,我需要从行中检索数据并将新数据附加到其中并将其写入同一行。这一定不是导致此问题的原因,因为这种情况在 200k 插入期间一定发生了很多次。

我正在使用外部硬盘来存储提交日志。concurrent_reads 设置为 16。 concurrent_writes 设置为 32。 read_timeout 已设置为 10000,write_timeout 也是如此。MAX_HEAP_SIZE="2G" HEAP_NEWSIZE="200M"

一旦我得到帽子异常,我就会看到这个带有这种模式的日志

    INFO [MemoryMeter:1] 2014-03-25 16:28:56,498 Memtable.java (line 449) CFS(Keyspace='rdf', ColumnFamily='lubm1m.lubm1m_teacherof_idx') liveRatio is 15.733535462912831 (just-counted was 15.733535462912831).  calculation took 91ms for 665 cells
 INFO [MemoryMeter:1] 2014-03-25 16:30:01,560 Memtable.java (line 449) CFS(Keyspace='system', ColumnFamily='sstable_activity') liveRatio is 15.896188158961882 (just-counted was 15.896188158961882).  calculation took 37ms for 411 cells
 INFO [ScheduledTasks:1] 2014-03-25 16:33:22,252 GCInspector.java (line 116) GC for ParNew: 15581 ms for 1 collections, 395151608 used; max is 1038876672
 INFO [ScheduledTasks:1] 2014-03-25 16:33:24,894 StatusLogger.java (line 55) Pool Name                    Active   Pending      Completed   Blocked  All Time Blocked
 INFO [ScheduledTasks:1] 2014-03-25 16:33:26,956 StatusLogger.java (line 70) ReadStage                         0         0         343094         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:26,957 StatusLogger.java (line 70) RequestResponseStage              0         0              1         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:26,996 StatusLogger.java (line 70) ReadRepairStage                   0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,004 StatusLogger.java (line 70) MutationStage                     0         0         601747         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,024 StatusLogger.java (line 70) ReplicateOnWriteStage             0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,031 StatusLogger.java (line 70) GossipStage                       0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,066 StatusLogger.java (line 70) AntiEntropyStage                  0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,067 StatusLogger.java (line 70) MigrationStage                    0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,098 StatusLogger.java (line 70) MemtablePostFlusher               0         0             52         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,099 StatusLogger.java (line 70) MemoryMeter                       0         0            139         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,100 StatusLogger.java (line 70) FlushWriter                       0         0             31         0                16
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,101 StatusLogger.java (line 70) MiscStage                         0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,130 StatusLogger.java (line 70) PendingRangeCalculator            0         0              1         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,133 StatusLogger.java (line 70) commitlog_archiver                0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,134 StatusLogger.java (line 70) InternalResponseStage             0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,142 StatusLogger.java (line 70) HintedHandoff                     0         0              0         0                 0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,223 StatusLogger.java (line 79) CompactionManager                 0         0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,232 StatusLogger.java (line 81) Commitlog                       n/a         0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,262 StatusLogger.java (line 93) MessagingService                n/a       0/0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,262 StatusLogger.java (line 103) Cache Type                     Size                 Capacity               KeysToSave
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,287 StatusLogger.java (line 105) KeyCache                      17212                 51380224                      all
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,335 StatusLogger.java (line 111) RowCache                          0                        0                      all
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,336 StatusLogger.java (line 118) ColumnFamily                Memtable ops,data
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,336 StatusLogger.java (line 121) system.schema_triggers                    0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,337 StatusLogger.java (line 121) system.local                              0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,337 StatusLogger.java (line 121) system.peers                              0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,338 StatusLogger.java (line 121) system.batchlog                           0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,338 StatusLogger.java (line 121) system.NodeIdInfo                         0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,339 StatusLogger.java (line 121) system.compactions_in_progress                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,339 StatusLogger.java (line 121) system.schema_keyspaces                4,1130
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.compaction_history            38,11220
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.paxos                              0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.schema_columns             3361,891818
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,341 StatusLogger.java (line 121) system.schema_columnfamilies         1426,394018
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,341 StatusLogger.java (line 121) system.IndexInfo                          0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,342 StatusLogger.java (line 121) system.range_xfers                        0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,344 StatusLogger.java (line 121) system.peer_events                        0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,344 StatusLogger.java (line 121) system.hints                              0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,345 StatusLogger.java (line 121) system.sstable_activity           1542,196000
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,345 StatusLogger.java (line 121) KeyspaceCumulus.POS_RN_NUM                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,346 StatusLogger.java (line 121) KeyspaceCumulus.DICT_SO_REVERSE                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,346 StatusLogger.java (line 121) KeyspaceCumulus.SPO_RN_DT                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,347 StatusLogger.java (line 121) KeyspaceCumulus.DICT_P_REVERSE                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,347 StatusLogger.java (line 121) KeyspaceCumulus.SPOC                      0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,348 StatusLogger.java (line 121) KeyspaceCumulus.POSC                      0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,348 StatusLogger.java (line 121) KeyspaceCumulus.counter                   0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,349 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_O_PROPS                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,350 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_D_PROPS                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,350 StatusLogger.java (line 121) KeyspaceCumulus.PREFIX_TO_NS                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,352 StatusLogger.java (line 121) KeyspaceCumulus.POS_RN_DT                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,352 StatusLogger.java (line 121) KeyspaceCumulus.SPO_RN_NUM                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,353 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_CLASSES                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,353 StatusLogger.java (line 121) KeyspaceCumulus.OSPC                      0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,354 StatusLogger.java (line 121) KeyspaceCumulus.DICT_SO                   0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,354 StatusLogger.java (line 121) KeyspaceCumulus.DICT_P                    0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,355 StatusLogger.java (line 121) rdf.animal                                0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,355 StatusLogger.java (line 121) rdf.lubm20                                0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,356 StatusLogger.java (line 121) rdf.bsbm50                                0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,356 StatusLogger.java (line 121) rdf.tim                                   0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,357 StatusLogger.java (line 121) rdf.bsbm250                               0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,357 StatusLogger.java (line 121) rdf.lubm1m                    152700,22140112
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,358 StatusLogger.java (line 121) rdf.lubm3                                 0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,358 StatusLogger.java (line 121) system_traces.sessions                    0,0
 INFO [ScheduledTasks:1] 2014-03-25 16:33:27,359 StatusLogger.java (line 121) system_traces.events                      0,0

可能是什么问题呢?

4

1 回答 1

1

使用 SocketOptions 增加套接字超时。您也可以尝试在 datastax 客户端和服务器端配置它。另外,看看降低一致性是否有帮助?

-维维克

于 2014-03-26T08:00:01.983 回答