hadoop - 为什么这段代码不会两次遍历 reducer 值？

Question

我有这个代码：

public void reduce(Text key, Iterable<Text> values, Context context) 
        throws IOException, InterruptedException 
        {
            String name = null; 
            String sid = null;
            String predicate = null;
            String oid = null;
            String id = null;
            String outKey = null;
            String outVal = null;

            LinkedList<Text> valuesList = new LinkedList<Text>();
            Iterator<Text> ite = values.iterator();
            while(ite.hasNext()) {
                Text t = ite.next();
                String[] entities = t.toString().split("#-#-#-#");
                        if(entities[entities.length-1].equalsIgnoreCase("topic_name"))
                {
                    name = entities[0];
                }
                valuesList.add(t);
            }
            Iterator<Text> ite2 = valuesList.iterator();
            while(ite2.hasNext()) { 
                Text t2 = ite2.next(); 
                String[] entities = t2.toString().split("#-#-#-#");
                if(!entities[entities.length-1].contains("topic_name"))
                {
                     if(name!=null) {
                     outKey = entities[0]+"\t"+entities[1]+"\t"+name;
                }
                else 
                {
                    outKey = entities[0]+"\t"+entities[1]+"\t"+key.toString();
                }
                context.write(new Text(outKey), null);
                }
            }
        }

我看到当我再次遍历这些值时，它总是取缓存副本中的最后一个值。

Answer 1

第一个迭代器实际上总是返回相同的Text对象，它只是在每次调用之前用不同的字符串填充它。这样做是为了节省实例化对象的时间。所以你实际上是在构建一个List<Text>包含同一对象的许多副本的。要解决此问题，您应该将值保存到List<String>包含实际“未装箱”值的 a 中。像这样：

        LinkedList<String> valuesList = new LinkedList<String>();
        Iterator<Text> ite = values.iterator();
        while(ite.hasNext()) {
            Text t = ite.next();
            String[] entities = t.toString().split("#-#-#-#");
                    if(entities[entities.length-1].equalsIgnoreCase("topic_name"))
            {
                name = entities[0];
            }
            valuesList.add(t.toString());
        }
        Iterator<String> ite2 = valuesList.iterator();
        while(ite2.hasNext()) { 
            String t2 = ite2.next(); 
            String[] entities = t2.split("#-#-#-#");
            if(!entities[entities.length-1].contains("topic_name"))
            {
                 if(name!=null) {
                 outKey = entities[0]+"\t"+entities[1]+"\t"+name;
            }
            else 
            {
                outKey = entities[0]+"\t"+entities[1]+"\t"+key.toString();
            }
            context.write(new Text(outKey), null);
            }
        }