python - python中的MapReduce解决方案识别不对称对

Question

我正在尝试用 Python 编写 Mapper 和 Reducer，这是一种我不熟悉的语言，它将打印非对称键值对。例如：输入

A,B
B,A
C,B
B,C
D,E
E,F
F,E
D,B

输出

D,E
D,B

Answer 1

>>> from collections import Counter
>>> pairs = [('A', 'B'), ('B', 'A'), ('C', 'B'), ('B', 'C'), ('D', 'E'), ('E', 'F'), ('F', 'E'), ('D', 'B')]
>>> [k for k, c in Counter(map(frozenset, pairs)).items() if c == 1]
[frozenset(['B', 'D']), frozenset(['E', 'D'])]

Answer 2

向映射器添加两个发射。

def mapper(record):
    key = record[0]
    value = record[1]
    mr.emit_intermediate(key, value)
    mr.emit_intermediate(value, key)

def reducer(key, list_of_values):
    for n in list_of_values:
        if list_of_values.count(n) < 2:        
            output = key, n
            mr.emit((output))