1

我是大数据的新手。现在我正在尝试将 JanusGraph 与 HBase 集成为后端。为此，我使用 gremlin 控制台。在这个控制台中，我写道：

JanusGraph graph = JanusGraphFactory.build().set("storage.backend", "hbase").open();

但我得到一个错误：

gremlin> JanusGraph graph = JanusGraphFactory.build().set("storage.backend", "hbase").open();
16:17:29 WARN  org.janusgraph.diskstorage.hbase.HBaseStoreManager  - Unexpected exception during getDeployment()
java.lang.RuntimeException: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
    at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:364)
    at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getFeatures(HBaseStoreManager.java:403)
    at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1377)
    at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
    at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
    at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:123)
    at org.janusgraph.core.JanusGraphFactory$Builder.open(JanusGraphFactory.java:264)
    at org.janusgraph.core.JanusGraphFactory$Builder$open$0.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
    at groovysh_evaluate.run(groovysh_evaluate:3)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
    at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
    at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:191)
    at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
    at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
    at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
    at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
    at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
    at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
    at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
    at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
    at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
    at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
    at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:166)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
    at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:478)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
    at org.janusgraph.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:739)
    at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getLocalKeyPartition(HBaseStoreManager.java:524)
    at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:361)
    ... 55 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the locations
    at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:319)
    at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
    at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
    at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
    at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
    at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
    at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:162)
    at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
    at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
    at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
    at org.janusgraph.diskstorage.hbase.HBaseAdmin1_0.tableExists(HBaseAdmin1_0.java:111)
    at org.janusgraph.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:716)
    ... 57 more

我不明白为什么会出现这个错误。hbase的配置文件：

<configuration>

    <property>
      <name>dfs.domain.socket.path</name>
      <value>/var/lib/hadoop-hdfs/dn_socket</value>
    </property>

    <property>
      <name>hbase.bulkload.staging.dir</name>
      <value>/apps/hbase/staging</value>
    </property>

    <property>
      <name>hbase.client.keyvalue.maxsize</name>
      <value>1048576</value>
    </property>

    <property>
      <name>hbase.client.retries.number</name>
      <value>35</value>
    </property>

    <property>
      <name>hbase.client.scanner.caching</name>
      <value>100</value>
    </property>

    <property>
      <name>hbase.cluster.distributed</name>
      <value>true</value>
    </property>

    <property>
      <name>hbase.coprocessor.master.classes</name>
      <value></value>
    </property>

    <property>
      <name>hbase.coprocessor.region.classes</name>
      <value>org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint</value>
    </property>

    <property>
      <name>hbase.custom-extensions.root</name>
      <value>/hdp/ext/2.6/hbase</value>
    </property>

    <property>
      <name>hbase.defaults.for.version.skip</name>
      <value>true</value>
    </property>

    <property>
      <name>hbase.hregion.majorcompaction</name>
      <value>604800000</value>
    </property>

    <property>
      <name>hbase.hregion.majorcompaction.jitter</name>
      <value>0.50</value>
    </property>

    <property>
      <name>hbase.hregion.max.filesize</name>
      <value>10737418240</value>
    </property>

    <property>
      <name>hbase.hregion.memstore.block.multiplier</name>
      <value>4</value>
    </property>

    <property>
      <name>hbase.hregion.memstore.flush.size</name>
      <value>134217728</value>
    </property>

    <property>
      <name>hbase.hregion.memstore.mslab.enabled</name>
      <value>true</value>
    </property>

    <property>
      <name>hbase.hstore.blockingStoreFiles</name>
      <value>10</value>
    </property>

    <property>
      <name>hbase.hstore.compaction.max</name>
      <value>10</value>
    </property>

    <property>
      <name>hbase.hstore.compactionThreshold</name>
      <value>3</value>
    </property>

    <property>
      <name>hbase.local.dir</name>
      <value>${hbase.tmp.dir}/local</value>
    </property>

    <property>
      <name>hbase.master.info.bindAddress</name>
      <value>0.0.0.0</value>
    </property>

    <property>
      <name>hbase.master.info.port</name>
      <value>16010</value>
    </property>

    <property>
      <name>hbase.master.namespace.init.timeout</name>
      <value>2400000</value>
    </property>

    <property>
      <name>hbase.master.port</name>
      <value>16000</value>
    </property>

    <property>
      <name>hbase.master.ui.readonly</name>
      <value>false</value>
    </property>

    <property>
      <name>hbase.master.wait.on.regionservers.timeout</name>
      <value>30000</value>
    </property>

    <property>
      <name>hbase.regionserver.executor.openregion.threads</name>
      <value>20</value>
    </property>

    <property>
      <name>hbase.regionserver.global.memstore.size</name>
      <value>0.4</value>
    </property>

    <property>
      <name>hbase.regionserver.handler.count</name>
      <value>30</value>
    </property>

    <property>
      <name>hbase.regionserver.info.port</name>
      <value>16030</value>
    </property>

    <property>
      <name>hbase.regionserver.port</name>
      <value>16020</value>
    </property>

    <property>
      <name>hbase.regionserver.wal.codec</name>
      <value>org.apache.hadoop.hbase.regionserver.wal.WALCellCodec</value>
    </property>

    <property>
      <name>hbase.rootdir</name>
      <value>hdfs://node01.main01.janus.apps.dev.int.grcc.labs.grcc.ru:8020/apps/hbase/data</value>
    </property>

    <property>
      <name>hbase.rpc.protection</name>
      <value>authentication</value>
    </property>

    <property>
      <name>hbase.rpc.timeout</name>
      <value>90000</value>
    </property>

    <property>
      <name>hbase.security.authentication</name>
      <value>simple</value>
    </property>

    <property>
      <name>hbase.security.authorization</name>
      <value>false</value>
    </property>

    <property>
      <name>hbase.superuser</name>
      <value>hbase</value>
    </property>

    <property>
      <name>hbase.tmp.dir</name>
      <value>/tmp/hbase-${user.name}</value>
    </property>

    <property>
      <name>hbase.zookeeper.property.clientPort</name>
      <value>2181</value>
    </property>

    <property>
      <name>hbase.zookeeper.quorum</name>
      <value>node01.main01.janus.apps.dev.int.grcc.labs.grcc.ru</value>
    </property>

    <property>
      <name>hbase.zookeeper.useMulti</name>
      <value>true</value>
    </property>

    <property>
      <name>hfile.block.cache.size</name>
      <value>0.4</value>
    </property>

    <property>
      <name>phoenix.query.timeoutMs</name>
      <value>60000</value>
    </property>

    <property>
      <name>zookeeper.recovery.retry</name>
      <value>6</value>
    </property>

    <property>
      <name>zookeeper.session.timeout</name>
      <value>90000</value>
    </property>

    <property>
      <name>zookeeper.znode.parent</name>
      <value>/hbase-unsecure</value>
    </property>
4

2 回答 2

1

在 中hbase-site.xml，请注意以下属性：

<property>
    <name>hbase.zookeeper.quorum</name>
    <value>node01.main01.janus.apps.dev.int.grcc.labs.grcc.ru</value>
</property>
<property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
</property>
<property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase-unsecure</value>
</property>

您应该使用这些来设置您的 JanusGraph 配置，否则将使用默认值。默认 znode/hbase与您的配置不同，因此这很可能会阻止连接。JanusGraph-HBase 文档中提到了如何设置它。我不清楚 Gremlin 控制台是否与 HBase 在同一台机器上运行，但默认主机是127.0.0.1. 默认端口是2181这样，即使它如下所示，也可以省略属性。

JanusGraph graph = JanusGraphFactory.build().
    set("storage.backend", "hbase").
    set("storage.hostname", "node01.main01.janus.apps.dev.int.grcc.labs.grcc.ru").
    set("storage.hbase.ext.zookeeper.property.clientPort", 2181).
    set("storage.hbase.ext.zookeeper.znode.parent", "/hbase-unsecure").
    open();
于 2017-11-27T16:13:00.573 回答
0

以下代码对我有用

Builder config = JanusGraphFactory.build();
config.set("storage.hbase.table", "student_table");
config.set("storage.backend", "hbase");
config.set("storage.hostname", "10.20.30.40");    
JanusGraph graph = config.open();

只有当我不使用弹性搜索时它才有效。

于 2018-05-22T13:01:04.687 回答