在我的映射器的任务日志中,我看到了这样的行为,这让我很困惑:
INFO org.apache.hadoop.mapred.Merger: Merging 6 sorted segments
INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 38872249 bytes
INFO org.apache.hadoop.mapred.Merger: Merging 6 sorted segments
INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 38636106 bytes
INFO org.apache.hadoop.mapred.Merger: Merging 6 sorted segments
INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 39490573 bytes
INFO org.apache.hadoop.mapred.Merger: Merging 6 sorted segments
....
repeat 20+ times
....
INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 37234005 bytes
INFO org.apache.hadoop.mapred.Task: Task:attempt_201209251414_0326_m_000052_0 is done. And is in the process of commiting
INFO org.apache.hadoop.mapred.Task: Task 'attempt_201209251414_0326_m_000052_0' done.\
我有 25 次以上的“最后一次”合并通行证是怎么回事?这个过程需要很长时间,所以我试图弄清楚我是否配置错误。