我经历了一次停电,导致我的 Cassandra 集群中的所有 4 个节点都死亡。我已经带回了所有的盒子,但是现在当我尝试启动 Cassandra(方法 bin/Cassandra)时,在重播日志后,我在 4 个中的 3 个上收到以下消息。
Exception encountered during startup: index (1) must be less than size (1)
INFO 10:11:00,479 CFS(Keyspace='system', ColumnFamily='peers') liveRatio is 13.10204081632653 (just-counted was 13.10204081632653). calculation took 18ms for 21 columns
(这只是结束,如果有帮助,我可以发布更多信息)
出现的一个仅用于读取,我假设因为我在所有键空间上都有复制因子 3,所以如果没有其他键空间它就无法工作。
关于如何检索情况的任何想法。我在数据库中有很多有价值的信息。
卡桑德拉 v1.2.6,
CentOs v6.4
更新:
在重播提交日志期间,所有三个死节点似乎都失败了。我将它们到达的行加粗。我最初的想法是删除该提交日志并尝试重新启动..但我有点害怕后果!
INFO 10:46:48,600 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208291.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208292.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208293.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208294.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208295.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208296.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208297.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208298.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208299.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208301.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208302.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208304.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208305.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208306.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208307.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208308.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208309.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208310.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208311.log, /var/lib/cassandra/commitlog/CommitLog-2-1381251642075.log, /var/lib/cassandra/commitlog/CommitLog-2-1381251731119.log
INFO 10:46:48,614 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208291.log
INFO 10:46:49,598 GC for ParNew: 605 ms for 2 collections, 50055912 used; max is 1046937600
INFO 10:46:51,775 Finished reading /var/lib/cassandra/commitlog/CommitLog-2-1379365208291.log
INFO 10:46:51,776 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208292.log
INFO 10:46:53,995 Finished reading /var/lib/cassandra/commitlog/CommitLog-2-1379365208292.log
INFO 10:46:53,995 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208293.log
**INFO 10:46:54,087 Finished reading /var/lib/cassandra/commitlog/CommitLog-2-1379365208293.log**
ERROR 10:46:54,088 Exception encountered during startup
java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:305)
at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:284)
at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:45)
at org.apache.cassandra.db.marshal.CompositeType.getComparator(CompositeType.java:94)
at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:76)
at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:102)
at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:88)
at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:114)
at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:101)
at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:397)
at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:202)
at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:97)
at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:146)
at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:126)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:298)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:484)
java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:305)
at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:284)
at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:45)
at org.apache.cassandra.db.marshal.CompositeType.getComparator(CompositeType.java:94)
at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:76)
at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:102)
at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:88)
at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:114)
at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:101)
at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:397)
at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:202)
at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:97)
at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:146)
at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:126)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:298)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:484)
Exception encountered during startup: index (1) must be less than size (1)
INFO 10:46:54,110 CFS(Keyspace='system', ColumnFamily='peers') liveRatio is 13.10204081632653 (just-counted was 13.10204081632653). calculation took 19ms for 21 columns