1

我正在测试一些基本的 HDFS 操作,比如创建目录。我的测试中有以下集群配置:

import org.apache.hadoop.fs._
import org.apache.hadoop.fs.permission.FsPermission
import org.apache.hadoop.hdfs.{HdfsConfiguration, MiniDFSCluster}

// ...

private val baseDir = new File("./target/hdfs/test").getAbsoluteFile

private val conf = new HdfsConfiguration()
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, baseDir.getAbsolutePath)
conf.setInt("dfs.safemode.threshold.pct", 0)
private val builder = new MiniDFSCluster.Builder(conf)
private val cluster = builder.build()
cluster.waitActive()
private val fs = cluster.getFileSystem

private val host = cluster.getNameNode.getHttpAddress.getHostString
private val port = cluster.getNameNodePort

我发现当我运行测试时,我发现我总是得到这个错误:

[warn] o.a.h.s.UserGroupInformation - PriviledgedActionException as:erip (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory [...]. Name node is in safe mode.
Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.

紧随其后的是...

[info]   org.apache.hadoop.ipc.RemoteException: Cannot create directory [...]. Name node is in safe mode.
[info] Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.

我正在运行一个内存集群,所以我不知道为什么我会看到这个。我认为设置"dfs.safemode.threshold.pct"会阻止我根据这个答案看到这个错误,但我错了。

为什么我的内存测试集群在安全模式下运行?我该如何阻止它这样做?

4

2 回答 2

0

使用以下命令在运行任何命令下方退出安全模式 -

hadoop dfsadmin -safemode leave

其他与安全模式相关的命令是

hadoop dfadmin -safemode get

获取系统当前运行状态

于 2018-02-14T04:58:13.223 回答
0

问题在于cluster.waitActive(),它等待名称节点准备好。这应该是cluster.waitClusterUp(),它明确地等待集群退出安全模式。

于 2018-02-14T21:14:01.290 回答