我的 Hadoop 机器上有一些损坏的文件,我想将它们传输到另一台计算机上,看看里面有什么。
我试着做hadoop fsck -copyToLocal /dir1/ /dir2/
。它什么也没给。当我做hadoop fs -copyToLocal /dir1/ /dir2/
或hadoop dfs -copyToLocal /dir1/ /dir2/
. 它写道:
/opt/atsd/hadoop/bin/hadoop fs -copyToLocal /hbase/ /home/axibase/Documents/new/
15/04/15 07:12:32 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59175, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:32 INFO hdfs.DFSClient: Could not obtain block blk_1601256999140141614_1170 from any node: java.io.IOException: No live nodes contain current block. Will get new block locations from namenode and retry...
15/04/15 07:12:35 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59176, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:35 INFO hdfs.DFSClient: Could not obtain block blk_1601256999140141614_1170 from any node: java.io.IOException: No live nodes contain current block. Will get new block locations from namenode and retry...
15/04/15 07:12:38 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59177, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:38 INFO hdfs.DFSClient: Could not obtain block blk_1601256999140141614_1170 from any node: java.io.IOException: No live nodes contain current block. Will get new block locations from namenode and retry...
15/04/15 07:12:41 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59178, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:41 WARN hdfs.DFSClient: DFS Read: java.io.IOException: Could not obtain block: blk_1601256999140141614_1170 file=/hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2269)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2063)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2224)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:47)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:87)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:341)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:248)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:199)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:1769)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
我也试着做hadoop fsck -move
。它会删除损坏的文件,但我找不到它们。