我已按照本网站的所有步骤进行操作。在我的 PC 上,一切都运行良好,但在我的笔记本电脑上 nodeManager 正在关闭。
以下是运行 start-all.cmd 命令后我的 nodeManager 状态。
r class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Res
ourceLocalizationService$LocalizerTracker
19/11/27 12:42:00 INFO containermanager.AuxServices: Adding auxiliary service ma
preduce_shuffle, "mapreduce_shuffle"
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: Using ResourceCalculatorP
lugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@2b72cb8a
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: Using ResourceCalculatorP
rocessTree : null
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: Physical memory check enab
led: true
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: Virtual memory check enabl
ed: true
19/11/27 12:42:00 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree curre
ntly is supported only on Linux.
19/11/27 12:42:00 INFO monitor.ContainersMonitorImpl: ContainersMonitor enabled:
true
19/11/27 12:42:01 WARN util.SysInfoWindows: Expected split length of sysInfo to
be 11. Got 7
19/11/27 12:42:01 WARN monitor.ContainersMonitorImpl: NodeManager's totalPmem co
uld not be calculated. Setting it to -1
19/11/27 12:42:01 INFO nodemanager.NodeStatusUpdaterImpl: Nodemanager resources:
memory set to 8192MB.
19/11/27 12:42:01 INFO nodemanager.NodeStatusUpdaterImpl: Nodemanager resources:
vcores set to 8.
19/11/27 12:42:01 INFO nodemanager.NodeStatusUpdaterImpl: Initialized nodemanage
r with : physical-memory=8192 virtual-memory=17204 virtual-cores=8
19/11/27 12:42:01 INFO util.JvmPauseMonitor: Starting JVM pause monitor
19/11/27 12:42:01 INFO ipc.CallQueueManager: Using callQueue: class java.util.co
ncurrent.LinkedBlockingQueue queueCapacity: 2000 scheduler: class org.apache.had
oop.ipc.DefaultRpcScheduler
19/11/27 12:42:01 INFO ipc.Server: Starting Socket Reader #1 for port 51483
19/11/27 12:42:01 WARN util.SysInfoWindows: Expected split length of sysInfo to
be 11. Got 7
19/11/27 12:42:01 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.had
oop.yarn.api.ContainerManagementProtocolPB to the server
19/11/27 12:42:01 INFO containermanager.ContainerManagerImpl: Blocking new conta
iner-requests as container manager rpc server is still starting.
19/11/27 12:42:01 INFO ipc.Server: IPC Server Responder: starting
19/11/27 12:42:01 INFO ipc.Server: IPC Server listener on 51483: starting
19/11/27 12:42:01 INFO security.NMContainerTokenSecretManager: Updating node add
ress : annupc:51483
19/11/27 12:42:01 INFO ipc.CallQueueManager: Using callQueue: class java.util.co
ncurrent.LinkedBlockingQueue queueCapacity: 500 scheduler: class org.apache.hado
op.ipc.DefaultRpcScheduler
19/11/27 12:42:01 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.had
oop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server
19/11/27 12:42:01 INFO localizer.ResourceLocalizationService: Localizer started
on port 8040
19/11/27 12:42:01 INFO ipc.Server: Starting Socket Reader #1 for port 8040
19/11/27 12:42:01 INFO ipc.Server: IPC Server listener on 8040: starting
19/11/27 12:42:01 INFO ipc.Server: IPC Server Responder: starting
19/11/27 12:42:01 WARN localizer.ResourceLocalizationService: Permissions incorr
ectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-xr-x,
actual value = rwxrwxr-x
19/11/27 12:42:01 INFO localizer.ResourceLocalizationService: Attempting to init
ialize /tmp/hadoop-anwar/nm-local-dir
19/11/27 12:42:02 WARN localizer.ResourceLocalizationService: Permissions incorr
ectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-xr-x,
actual value = rwxrwxr-x
19/11/27 12:42:02 WARN localizer.ResourceLocalizationService: Failed to setup lo
cal dir /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions incorrectly
set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-xr-x, actua
l value = rwxrwxr-x
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
19/11/27 12:42:02 INFO service.AbstractService: Service org.apache.hadoop.yarn.s
erver.nodemanager.containermanager.localizer.ResourceLocalizationService failed
in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
Failed to setup local dir /tmp/hadoop-anwar/nm-local-dir, which was marked as g
ood.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
... 13 more
19/11/27 12:42:02 INFO ipc.Server: Stopping server on 8040
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server listener on 8040
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server Responder
19/11/27 12:42:02 INFO localizer.ResourceLocalizationService: Public cache exiti
ng
19/11/27 12:42:02 INFO service.AbstractService: Service org.apache.hadoop.yarn.s
erver.nodemanager.containermanager.ContainerManagerImpl failed in state STARTED;
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup
local dir /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
... 13 more
19/11/27 12:42:02 INFO ipc.Server: Stopping server on 51483
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server listener on 51483
19/11/27 12:42:02 INFO ipc.Server: Stopping IPC Server Responder
19/11/27 12:42:02 INFO service.AbstractService: Service NodeManager failed in st
ate STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Fail
ed to setup local dir /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
... 13 more
19/11/27 12:42:02 WARN nodemanager.NodeResourceMonitorImpl: org.apache.hadoop.ya
rn.server.nodemanager.NodeResourceMonitorImpl is interrupted. Exiting.
19/11/27 12:42:02 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics syst
em...
19/11/27 12:42:02 INFO impl.MetricsSystemImpl: NodeManager metrics system stoppe
d.
19/11/27 12:42:02 INFO impl.MetricsSystemImpl: NodeManager metrics system shutdo
wn complete.
19/11/27 12:42:02 FATAL nodemanager.NodeManager: Error starting NodeManager
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local di
r /tmp/hadoop-anwar/nm-local-dir, which was marked as good.
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1533)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:271
)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.registe
rDirsChangeListener(DirectoryCollection.java:197)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.reg
isterLocalDirsChangeListener(LocalDirsHandlerService.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.serviceStart(ResourceLocalizationService.java:371)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerM
anagerImpl.serviceStart(ContainerManagerImpl.java:490)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeServ
ice.java:120)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(No
deManager.java:369)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:
193)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
eManager(NodeManager.java:637)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
r.java:684)
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions i
ncorrectly set for dir /tmp/hadoop-anwar/nm-local-dir/usercache, should be rwxr-
xr-x, actual value = rwxrwxr-x
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1560)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationServ
ice.java:1528)
... 13 more
19/11/27 12:42:02 INFO nodemanager.NodeManager: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NodeManager at annupc/192.168.1.103
************************************************************/