我扩展了 WritableComparable 并希望将其存储为映射器作为映射器值。
public class SenderRecieverPair implements WritableComparable<BinaryComparable> {
Set<InternetAddress> pair = new TreeSet<InternetAddress>(new Comparator<InternetAddress>() {
@Override
public int compare(InternetAddress add1, InternetAddress add2) {
return add1.getAddress().compareToIgnoreCase(add2.getAddress());
}
});
public SenderRecieverPair() {
super();
}
public SenderRecieverPair(InternetAddress add1, InternetAddress add2) {
super();
pair.add(add1);
pair.add(add1);
}
public Set<InternetAddress> getPair() {
return pair;
}
@Override
public void write(DataOutput out) throws IOException {
for (Iterator<InternetAddress> iterator = pair.iterator(); iterator.hasNext();) {
InternetAddress email = (InternetAddress) iterator.next();
String mailAddress = email.getAddress();
if(mailAddress == null) {
mailAddress = "";
}
byte[] address = mailAddress.getBytes("UTF-8");
WritableUtils.writeVInt(out, address.length);
out.write(address, 0, address.length);
String displayName = email.getPersonal();
if(displayName == null) {
displayName = "";
}
byte[] display = displayName.getBytes("UTF-8");
WritableUtils.writeVInt(out, display.length);
out.write(display, 0, display.length);
}
}
@Override
public void readFields(DataInput in) throws IOException {
for (int i = 0; i < 2; i++) {
int length = WritableUtils.readVInt(in);
byte[] container = new byte[length];
in.readFully(container, 0, length);
String mailAddress = new String(container, "UTF-8");
length = WritableUtils.readVInt(in);
container = new byte[length];
in.readFully(container, 0, length);
String displayName = new String(container, "UTF-8");
InternetAddress address = new InternetAddress(mailAddress, displayName);
pair.add(address);
}
}
@Override
public int compareTo(BinaryComparable o) {
// TODO Auto-generated method stub
return 0;
}
}
但是我收到以下错误。请帮助我理解并纠正这一点
2013-07-29 06:49:26,753 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
2013-07-29 06:49:26,891 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId=
2013-07-29 06:49:27,004 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100
2013-07-29 06:49:27,095 INFO org.apache.hadoop.mapred.MapTask: data buffer = 79691776/99614720
2013-07-29 06:49:27,095 INFO org.apache.hadoop.mapred.MapTask: record buffer = 262144/327680
2013-07-29 06:49:27,965 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2013-07-29 06:49:27,988 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-07-29 06:49:27,991 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: java.io.EOFException
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:128)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:967)
at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1253)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:581)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:648)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:250)
at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:299)
at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:320)
at com.edureka.sumit.enron.datatype.SenderRecieverPair.readFields(SenderRecieverPair.java:68)
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:122)
... 14 more
2013-07-29 06:49:27,993 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
谢谢