java - Cloudera Manager 4.0 字数统计示例

Question

我安装了 Cloudera Manager 并有 2 个节点连接到一个主节点。我正在尝试使用 cloudera 在此链接上提供的教程运行 wordcount 程序： https ://ccp.cloudera.com/display/SUPPORT/Hadoop+Tutorial#HadoopTutorial-InputsandOutputs

Cloudera 说要编译我在命令提示符下使用 nano 创建的 WordCount.java 文件。我创建了 Wordcount_Classes 目录并将 wordcount.java 文件放在那里。我不知道该目录是否应该移动到 cloudera 目录之一中的某个位置以使 java 代码工作？或者，如果我做错了什么。我安装了 Cloudera Manager，它已经在所有节点上自动安装了 ClouderaHadoop。我在我的终端上得到这个：

mark@*****:~/wordcount_classes$ javac WordCount.java
WordCount.java:6: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.Path;
                           ^
WordCount.java:7: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
WordCount.java:8: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
WordCount.java:9: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.*;
^
WordCount.java:10: package org.apache.hadoop.util does not exist
import org.apache.hadoop.util.*;
^
WordCount.java:14: cannot find symbol
symbol  : class MapReduceBase
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                  ^
WordCount.java:14: cannot find symbol
symbol  : class Mapper
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                           ^
WordCount.java:14: cannot find symbol
symbol  : class LongWritable
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                  ^
WordCount.java:14: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                ^
WordCount.java:14: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                      ^
WordCount.java:14: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                            ^
WordCount.java:14: interface expected here
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                 ^
WordCount.java:15: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Map
    private final static IntWritable one = new IntWritable(1);
                         ^
WordCount.java:16: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    private Text word = new Text();
            ^
WordCount.java:18: cannot find symbol
symbol  : class LongWritable
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                    ^
WordCount.java:18: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                      ^
WordCount.java:18: cannot find symbol
symbol  : class OutputCollector
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                  ^
WordCount.java:18: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                  ^
WordCount.java:18: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                        ^
WordCount.java:18: cannot find symbol
symbol  : class Reporter
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                             ^
WordCount.java:28: cannot find symbol
symbol  : class MapReduceBase
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                     ^
WordCount.java:28: cannot find symbol
symbol  : class Reducer
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                              ^
WordCount.java:28: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                      ^
WordCount.java:28: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                            ^
WordCount.java:28: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                         ^
WordCount.java:28: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                               ^
WordCount.java:28: interface expected here
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                     ^
WordCount.java:29: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                       ^
WordCount.java:29: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                          ^
WordCount.java:29: cannot find symbol
symbol  : class OutputCollector
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                               ^
WordCount.java:29: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                               ^
WordCount.java:29: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                     ^
WordCount.java:29: cannot find symbol
symbol  : class Reporter
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                                          ^
WordCount.java:15: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Map
    private final static IntWritable one = new IntWritable(1);
                                               ^
WordCount.java:16: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    private Text word = new Text();
                            ^
WordCount.java:32: operator + cannot be applied to int,IntWritable.get
        sum += values.next().get();
            ^
WordCount.java:32: inconvertible types
found   : <nulltype>
required: int
        sum += values.next().get();
                                ^
WordCount.java:34: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Reduce
      output.collect(key, new IntWritable(sum));
                              ^
WordCount.java:39: cannot find symbol
symbol  : class JobConf
location: class org.myorg.WordCount
    JobConf conf = new JobConf(WordCount.class);
    ^
WordCount.java:39: cannot find symbol
symbol  : class JobConf
location: class org.myorg.WordCount
    JobConf conf = new JobConf(WordCount.class);
                       ^
WordCount.java:42: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
    conf.setOutputKeyClass(Text.class);
                           ^
WordCount.java:43: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
    conf.setOutputValueClass(IntWritable.class);
                             ^
WordCount.java:49: cannot find symbol
symbol  : class TextInputFormat
location: class org.myorg.WordCount
    conf.setInputFormat(TextInputFormat.class);
                        ^
WordCount.java:50: cannot find symbol
symbol  : class TextOutputFormat
location: class org.myorg.WordCount
    conf.setOutputFormat(TextOutputFormat.class);
                         ^
WordCount.java:52: cannot find symbol
symbol  : class Path
location: class org.myorg.WordCount
    FileInputFormat.setInputPaths(conf, new Path(args[0]));
                                            ^
WordCount.java:52: cannot find symbol
symbol  : variable FileInputFormat
location: class org.myorg.WordCount
    FileInputFormat.setInputPaths(conf, new Path(args[0]));
    ^
WordCount.java:53: cannot find symbol
symbol  : class Path
location: class org.myorg.WordCount
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));
                                             ^
WordCount.java:53: cannot find symbol
symbol  : variable FileOutputFormat
location: class org.myorg.WordCount
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));
    ^
WordCount.java:55: cannot find symbol
symbol  : variable JobClient
location: class org.myorg.WordCount
    JobClient.runJob(conf);
    ^
49 errors

有人可以帮我运行 wordcount 程序或指导我应该做什么，或者如果有人知道我找不到的在线教程吗？谢谢！

score 2 · Accepted Answer

您需要您的类路径上的依赖项（对应于您的导入语句）javac。

hadoop classpath使用 Hadoop，通过实用程序更容易做到这一点：

$ javac -classpath `hadoop classpath`:. WordCount.java

cannot find symbol正确设置类路径后，您不应该遇到任何此类编译问题。

java - Cloudera Manager 4.0 字数统计示例

1 回答 1

Related

Reference