我想传递一个浮点数组作为映射器中的键,
Such as [0.6, 0, 10]
我使用 StringUtils.join 将其转换为字符串,然后编写
context.write(new Text(str), value);
但是在减速器类中,我发现我得到的是:
"0 0 1"
如何解决?我正在使用 hadoop 1.0.4
我想传递一个浮点数组作为映射器中的键,
Such as [0.6, 0, 10]
我使用 StringUtils.join 将其转换为字符串,然后编写
context.write(new Text(str), value);
但是在减速器类中,我发现我得到的是:
"0 0 1"
如何解决?我正在使用 hadoop 1.0.4
您可以使用 . 发出浮点数组ArrayWritable
。我不确定将数组作为键发出,下面是您如何发出与值相同的值。
ArrayWritable does not implement WritableComparable, so it can't currently
be used as a mapper output key - those keys have to be sorted during the
shuffle, and thus the type must be WritableComparable.
通常我们使用自定义的 Writable Comparable Class 来发射键。
为此,您应该在驱动程序类 public class Driver {
public static class DoubleArrayWritable extends ArrayWritable {
public DoubleArrayWritable() {
super(DoubleWritable.class);
}
}
public static void main(String[] args) {
在映射器中
DoubleArrayWritable pointArray = new DoubleArrayWritable();
DoubleWritable[] data = new DoubleWritable[numAttributes];
for (int k = 0; k < numAttributes; k++) {
data[k] = new DoubleWritable(point[k]);
}
pointArray.set(data);
context.write(new Text("Array"), pointArray);
希望这在某些方面有所帮助。