2

我正在使用 Ubuntu-12.04,Hadoop-1.0.2,Hive-0.10.0 从 hive 读取大约 100 万条记录的数据时出现以下错误以进行查询

select * from raw_pos limit 10000;
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

所以我在 $HADOOP_HOME/lib 文件夹中安装了 Snappy for Hadoop，它会生成文件 libsnappy.a、libsnappy.la、libsnappy.so、libsnappy.so.1、libsnappy.so.1.1.4

还在 $HADOOP_HOME/lib/ 中添加 hadoop-lzo-0.4.3.jar 并在 cor-site.xml,mapred-site.xml 中进行如下更改

核心站点.xml：-

<property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:54310</value>
</property>

<property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/apache/hadoop-1.0.4/hadoop_temp/</value>
    <description>A base for other temporary directories.</description>
</property>

<property>
<name>io.compression.codecs</name>
<value>
  org.apache.hadoop.io.compress.SnappyCodec
</value>

mapred-site.xml :-

<property>
    <name>mapred.job.tracker</name>
    <value>hdfs://localhost:54311</value>
</property>

<property>
  <name>mapred.compress.map.output</name>
 <value>true</value>
</property>

<property>
   <name>mapred.map.output.compression.codec</name>
   <value>org.apache.hadoop.io.compress.SnappyCodec</value>
</property>

但是当我开始 hive & do 时show databases，会出现错误：-

Failed with exception java.io.IOException:java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
4

1 回答 1

1

将您的 core-site.xml 修改为此，看看它是否有帮助：

<property>
<name>io.compression.codecs</name>
<value>com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.SnappyCodec</value>
</property>

<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
于 2013-06-10T20:18:58.220 回答