hadoop - 如何通过（b，a）过滤（a，b）关系？

Question

我有一个像这样的通用关系 A：

DUMP A;
(a, b)
(a, c)
(a, d)
(b, a)
(d, a)
(d, b)

看到有 (a,b) 和 (b,a) 对；但是（d，b）没有一对。我想过滤掉那些“不成对”的元组。

最终结果应该是这样的：

DUMP R; 
(a, b)
(a, d)
(b, a)
(d, a)

我怎样才能在 PIG 上写这个？

我可以用下面的代码解决，但是交叉操作太贵了：

A_cp = FOREACH L GENERATE u1, u2;
X = CROSS A, A_cp;
F = FILTER X BY ($0 == $3 AND $1 == $2);
R = FOREACH F GENERATE $0, $1;

Answer 1

这是我的输出DESCRIBE A ; DUMP A ;：

A: {first: chararray,second: chararray}
(a,b)
(a,c)
(a,d)
(b,a)
(d,a)
(d,b)

这是您可以解决此问题的一种方法：

A = LOAD 'foo.in' AS (first:chararray, second:chararray) ;
-- Can't do a join on its self, so we have to duplicate A
A2 = FOREACH A GENERATE * ;

-- Join the As so that are in (b,a,a,c) etc. pairs.
B = JOIN A BY second, A2 BY first ; 

-- We only want pairs where the first char is equal to the last char.
C = FOREACH (FILTER B BY A::first == A2::second)
    -- Now we project out just one side of the pair.
    GENERATE A::first AS first, A::second AS second ;

输出：

C: {first: chararray,second: chararray}
(b,a)
(d,a)
(a,b)
(a,d)

更新：正如 WinnieNicklaus 指出的，这可以缩短为：

B = FOREACH (JOIN A BY (first, second), A2 BY (second, first))
    GENERATE A::first AS first, A::second AS second ;