我是新来的火花,我正在“不等于”条件下加入两个数据帧。在某一时刻,我的程序停止并且不再运行,也没有给出异常。
我正在使用一个包含 100000 条记录的简单文本文件。
我的程序中有一个非惰性方法“collectAsList”,它会导致执行连接。
这是堆栈跟踪:
15/10/14 09:25:36 INFO TaskSchedulerImpl: Adding task set 25.0 with 2 tasks
15/10/14 09:25:38 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on 192.168.1.38:50065 (size: 4.7 KB, free: 5.2 GB)
15/10/14 09:25:38 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 192.168.1.38:50065 (size: 4.8 KB, free: 5.2 GB)
15/10/14 09:25:38 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 192.168.1.37:48062 (size: 4.8 KB, free: 5.2 GB)
15/10/14 09:25:38 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on 192.168.1.37:48062 (size: 4.7 KB, free: 5.2 GB)
15/10/14 09:25:39 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 192.168.1.38:50065 (size: 13.9 KB, free: 5.2 GB)
15/10/14 09:25:39 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 192.168.1.37:48062 (size: 13.9 KB, free: 5.2 GB)