3

我正在尝试运行 Giraph 中包含的 SimpleInDegreeCountComputation 示例。我的方法如下：

SimpleInDegreeCountComputation.java:

    public class SimpleInDegreeCountComputation extends BasicComputation
              <LongWritable, LongWritable, DoubleWritable, DoubleWritable> {
    .......

然后我尝试像这样运行它：

    hadoop jar /path-to-giraph-folder/giraph-examples/target/giraph-examples-1.1.0- 
    SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar 
    org.apache.giraph.GiraphRunner  
    org.apache.giraph.examples.SimpleInDegreeCountComputation 
    -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat 
    -vip /path-to-input-file 
    -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat 
    -op /path-to-output-file -w 1

结果如下：

    14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge input format specified. 
    Ensure your InputFormat does not require one.
    14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge output format specified.  
    Ensure your OutputFormat does not require one.
    Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: vertex  
    value types not assignable, computation - class org.apache.hadoop.io.LongWritable,   
    VertexInputFormat - class org.apache.hadoop.io.DoubleWritable
at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

我不太确定我做错了什么。如果有人能指出我正确的方向，或者链接到一个资源来解释我正在尝试做的更简单的方法，我将不胜感激！我认为问题可能是错误的格式（-vif）。我正在使用的输入文件如下：

    [0,0,[[1,5],[2,9]]]
    [1,0,[[0,5],[3,3]]]
    [2,0,[[0,9],[3,3],[4,3]]]
    [3,0,[[1,3],[2,3],[4,2]]]
    [4,0,[[2,3],[3,3]]]
4

1 回答 1

6

查看计算和顶点输入类的定义，似乎JsonLongDoubleFloatDoubleVertexInputFormat不兼容SimpleInDegreeCountComputation

SimpleInDegreeCountComputation：

public class SimpleInDegreeCountComputation extends BasicComputation<
    LongWritable, LongWritable, DoubleWritable, DoubleWritable> {

基本计算：

/**
 * Computation in which both incoming and outgoing message types are the same.
 *
 * @param <I> Vertex id
 * @param <V> Vertex data
 * @param <E> Edge data
 * @param <M> Message type
 */
public abstract class BasicComputation<I extends WritableComparable,
    V extends Writable, E extends Writable, M extends Writable>
    extends AbstractComputation<I, V, E, M, M> {
}

你可以看到：

  • 顶点 id 的类型LongWritable
  • 顶点数据的类型LongWritable
  • 边缘数据的类型DoubleWritable

...另一方面，您尝试使用的 InputFormat ...

JsonLongDoubleFloatDoubleVertexInputFormat：

public class JsonLongDoubleFloatDoubleVertexInputFormat extends
    TextVertexInputFormat<LongWritable, DoubleWritable, FloatWritable> {

文本顶点输入格式：

/**
 * Abstract class that users should subclass to use their own text based
 * vertex input format.
 *
 * @param <I> Vertex index value
 * @param <V> Vertex value
 * @param <E> Edge value
 */
@SuppressWarnings("rawtypes")
public abstract class TextVertexInputFormat<I extends WritableComparable,
    V extends Writable, E extends Writable>
    extends VertexInputFormat<I, V, E> {

你可以看到：

  • 顶点 id 的类型LongWritable
  • 顶点数据的类型DoubleWritable
  • 边缘数据的类型FloatWritable

由于它是LongWritable, DoubleWritableand FloatWritableand not Long, Doubleand Float- 这些类型不能自动转换。

我找不到您可以使用的任何 InputFormat，因此您需要修改现有的JsonLongDoubleFloatDoubleVertexInputFormat或修改算法以NullWritable用于 Edge 数据类型。我在任何地方都看不到要使用的边缘数据，因此它也可以为空。在这种情况下，您可以使用LongLongNullTextInputFormat。

于 2014-05-18T19:20:36.840 回答