3

我的配置：

  • CouchBase 服务器 2.0
  • Sqoop 1.4.2（为 hadoop 版本 2.0.0 编译）
  • 堆栈 Hadoop CDH4.1.2

我想使用 CouchBase/Hadoop 连接器（http://www.couchbase.com/develop/connectors/hadoop），但是当我启动导入时出现以下错误：

sqoop-import --connect http://localhost:8091/pools --table DUMP
13/01/07 11:00:34 INFO tool.CodeGenTool: Beginning code generation
13/01/07 11:00:34 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop
Note: /tmp/sqoop-agent/compile/fccb413cf9eec92f9a716320cf268a0a/DUMP.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/01/07 11:00:42 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-agent/compile/fccb413cf9eec92f9a716320cf268a0a/DUMP.jar
13/01/07 11:00:42 INFO mapreduce.ImportJobBase: Beginning import of DUMP
13/01/07 11:00:45 WARN util.Jars: No such class couchbase doesn't use a jdbc driver available.
13/01/07 11:00:50 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2013-01-07 11:00:55.065 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-07 11:00:55.098 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@41e22632
2013-01-07 11:00:56.266 INFO net.spy.memcached.MemcachedConnection:  Shut down memcached client
13/01/07 11:02:39 INFO mapred.JobClient: Running job: job_201212111702_0131
13/01/07 11:02:40 INFO mapred.JobClient:  map 0% reduce 0%
13/01/07 11:02:59 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000001_0,     Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:02 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000002_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:02 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000000_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:09 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000003_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:12 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000001_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:13 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000002_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:20 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000000_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:24 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000003_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:24 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000001_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:30 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000002_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:33 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000000_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:39 INFO mapred.JobClient: Job complete: job_201212111702_0131
13/01/07 11:03:40 INFO mapred.JobClient: Counters: 7
13/01/07 11:03:40 INFO mapred.JobClient:   Job Counters 
13/01/07 11:03:40 INFO mapred.JobClient:     Failed map tasks=1
13/01/07 11:03:40 INFO mapred.JobClient:     Launched map tasks=14
13/01/07 11:03:40 INFO mapred.JobClient:     Rack-local map tasks=14
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all maps in occupied slots (ms)=74621
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all reduces in occupied slots (ms)=0
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/01/07 11:03:40 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 174.4617 seconds (0 bytes/sec)
13/01/07 11:03:40 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/01/07 11:03:40 INFO mapreduce.ImportJobBase: Retrieved 0 records.
13/01/07 11:03:40 ERROR tool.ImportTool: Error during import: Import job failed!

有没有人使用过具有相同配置的连接器并成功使其工作？

谢谢你的帮助！

玛德琳

4

2 回答 2

3

Hadoop 经历了从 Hadoop 1.0 到 Hadoop 2.0（相应地从 CDH3 到 CDH4）的巨大代码重构。一个副作用是针对 Hadoop 1.0 (CDH3) 编译的代码与 Hadoop 2.0 (CDH4) 不兼容，反之亦然。但是源代码是兼容的，因此只需要使用目标 Hadoop 发行版重新编译代码。

当您在 Hadoop 2.0 (CDH4) 上运行为 Hadoop 1.0 (CDH3) 编译的代码时，异常“找到接口 X，但预期类”非常常见，反之亦然。

完全相同的约束也适用于连接器。没有针对哪个 Hadoop 版本 CouchBase 连接器编译的明确信息，所以我猜它是针对 Hadoop 1.0 的，因此它不适用于 Hadoop 2.0 (CDH4)。

于 2013-01-09T08:02:14.893 回答
0

这已在 CDH4.2 版本中得到修复。

http://www.cloudera.com/content/cloudera/en/documentation/cdh4/v4-2-0/CDH4-Release-Notes/cdh4ki_topic_2_9.html

于 2015-04-20T02:00:57.470 回答