3

I'm trying to back up a directory from hdfs to a local directory. I have a hadoop/hbase cluster running on ec2. I managed to do what I want running in pseudo-distributed on my local machine but now I'm fully distributed the same steps are failing. Here is what worked for pseudo-distributed

hadoop distcp hdfs://localhost:8020/hbase file:///Users/robocode/Desktop/

Here is what I'm trying on the hadoop namenode (hbase master) on ec2

ec2-user@ip-10-35-53-16:~$ hadoop distcp hdfs://10.35.53.16:8020/hbase file:///~/hbase

The errors I'm getting are below

13/04/19 09:07:40 INFO tools.DistCp: srcPaths=[hdfs://10.35.53.16:8020/hbase]
13/04/19 09:07:40 INFO tools.DistCp: destPath=file:/~/hbase
13/04/19 09:07:41 INFO tools.DistCp: file:/~/hbase does not exist.
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.IOException: Failed to createfile:/~/hbase
    at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1171)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:666)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
4

1 回答 1

6

Java 中不能使用 ~ 字符来表示当前主目录，所以改为完全限定路径，例如：

file:///home/user1/hbase

但我认为你会在完全分布式环境中遇到问题，因为 distcp 命令运行 map reduce 作业，因此目标路径将被解释为每个集群节点的本地路径。

如果要将数据从 HDFS 拉到本地目录，则需要使用 -get 或 -copyToLocal 开关到hadoop fs命令

于 2013-04-19T11:29:30.037 回答