0

我对 Hbase 很陌生。我在独立模式下使用 Hbase 并使用 google n-grams 开发应用程序。我在 Amazon Ec2 实例中使用 ubuntu 框

我正在通过批处理 api 使用我的程序进行 100 万次写入。

        HTable table = new HTable(conf, tableName);
        conf.set("zookeeper.session.timeout", "180000");
        conf.set("hbase.rpc.timeout", "180000");
        conf.set("hbase.regionserver.logroll.period", "180000");
        conf.set("hbase.hregion.max.filesize","51200");
        table.setAutoFlush(false);
        table.batch(iterateList);
        table.flushCommits();

当我用 1M 行执行此操作时，我按以下顺序收到以下异常，

    12/08/28 14:17:13 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1396d9653bd0004, likely server has closed socket, closing socket connection and attempting reconnect
    12/08/28 14:17:13 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1396d9653bd0003, likely server has closed socket, closing socket connection and attempting reconnect
    12/08/28 14:17:13 WARN client.HConnectionManager$HConnectionImplementation: Failed all from region=googlegrams,,1346160551634.abc8baf335dee8485d64a6c8b989d6b0., hostname=ip-10-80-74-132.ec2.internal, port=45549

java.util.concurrent.ExecutionException：java.io.IOException：调用 ip-10-80-74-132.ec2.internal/10.80.74.132:45549 本地异常失败：java.io.EOFException

接着

  12/08/28 14:17:15 WARN zookeeper.ClientCnxn: Session 0x1396d9653bd0004
  for server null, unexpected error, closing socket connection and attempting reconnect
  java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)

我尝试过的几件事是，1. 更改 ubuntu 的 ulimit 和 nproc 设置 2. Hbase 配置，例如 zookeeper 超时、rpc 超时 logroll period 和 hregion fileSize

请帮忙！

干杯，德瓦拉克

4

1 回答 1

1

尝试增加 hbase.client.lease.period

于 2012-10-28T10:35:16.653 回答