0

我想在有 HA 的集群上部署 Alluxio。我的 CDH 版本:3.0.0+cdh6.3.2。

我使用特定的 Hadoop 发布版本构建 Alluxio:

mvn install -Phadoop-3 -Dhadoop.version=3.0.0 -DskipTests

我将 alluxio-assembly-server-2.4.1-2-SNAPSHOT-jar-with-dependencies.jar 和 alluxio-underfs-hdfs-2.4.1-2-SNAPSHOT-jar-with-dependencies.jar 放在 lib/ 文件夹中Alluxio 的每个节点。

/opt/alluxio-2.4.1-1/conf/alluxio-site.properties:

alluxio.master.mount.table.root.ufs=hdfs://nameservice1/alluxio/data
alluxio.master.journal.type=UFS
alluxio.master.journal.folder=hdfs://nameservice1/alluxio/journal/
alluxio.master.security.impersonation.root.users=*
alluxio.worker.tieredstore.level0.dirs.quota=10GB
alluxio.worker.tieredstore.level1.dirs.quota=10GB
alluxio.worker.tieredstore.level2.dirs.quota=10GB

alluxio.zookeeper.enabled=true
alluxio.zookeeper.address=test-cdh001:2181,test-cdh002:2181,test-cdh003:2181

alluxio.underfs.hdfs.configuration=/etc/hadoop/conf/core-site.xml:/etc/hadoop/conf/hdfs-site.xml

当我在其中一个主节点中使用以下命令格式化 Alluxio 集群时:

./bin/alluxio format

我收到一个错误:

Executing the following command on all worker nodes and logging to /opt/alluxio-2.4.1-1/logs/task.log: /opt/alluxio-2.4.1-1/bin/alluxio formatWorker
Waiting for tasks to finish...
All tasks finished
Formatting Alluxio Master @ test-cdh001
2021-01-07 18:35:58,766 INFO  Format - Formatting master journal: hdfs://nameservice1/alluxio/journal/
2021-01-07 18:35:58,806 INFO  ExtensionFactoryRegistry - Loading core jars from /opt/alluxio-2.4.1-1/lib
2021-01-07 18:35:58,869 INFO  ExtensionFactoryRegistry - Loading extension jars from /opt/alluxio-2.4.1-1/extensions
2021-01-07 18:35:58,886 WARN  ExtensionFactoryRegistry - No factory implementation supports the path hdfs://nameservice1/alluxio/journal/BlockMaster
2021-01-07 18:35:58,887 INFO  ExtensionFactoryRegistry - Loading core jars from /opt/alluxio-2.4.1-1/lib
2021-01-07 18:35:58,906 INFO  ExtensionFactoryRegistry - Loading extension jars from /opt/alluxio-2.4.1-1/extensions
2021-01-07 18:35:58,915 WARN  ExtensionFactoryRegistry - No factory implementation supports the path hdfs://nameservice1/alluxio/journal/BlockMaster
2021-01-07 18:35:58,915 ERROR Format - Failed to format
java.lang.IllegalArgumentException: No Under File System Factory found for: hdfs://nameservice1/alluxio/journal/BlockMaster
    at alluxio.underfs.UnderFileSystem$Factory.create(UnderFileSystem.java:95)
    at alluxio.master.journal.ufs.UfsJournal.<init>(UfsJournal.java:149)
    at alluxio.master.journal.ufs.UfsJournalSystem.createJournal(UfsJournalSystem.java:73)
    at alluxio.master.journal.ufs.UfsJournalSystem.createJournal(UfsJournalSystem.java:47)
    at alluxio.cli.Format.format(Format.java:120)
    at alluxio.cli.Format.main(Format.java:97)

任何帮助将非常感激。

4

1 回答 1

0

使用 "--option alluxio.underfs.version=" 将您的 hadoop 版本添加到 mount 命令

前任。alluxio fs mount --option alluxio.underfs.version=3.2 /mnt/hdfs/emr hdfs://hostname:8020/tmp/emr

于 2021-02-02T22:51:27.420 回答