1

我有一个与流作业相关联的简单查询，其中没有 order by 语句。

set hive.exec.max.dynamic.partitions.pernode=100;
set hive.exec.max.dynamic.partitions=100;
set hive.exec.max.created.files=100;
set hive.exec.dynamic.partition.mode=nonstrict;
set mapred.reduce.tasks=20;
add file /home/devo/c1166313/pafvalid.py ;
add file /home/devo/c1166313/paf-rules.properties ;
from
 (from  
   (select * from mz_paf_errors_dummy_v) p
select transform (p.*)  row format delimited fields terminated by '|' 
using 'pafvalid.py paf-rules.properties 10'
as (<column list>)
row format delimited fields terminated by '|' )  b
insert overwrite table mytab partition (passfail, batch_sk) select <col list>;

这是一个中等规模的集群（几十台机器），映射器的数量超过两千。为什么需要一个减速器？

Number of reduce tasks determined at compile time: 1
4

0 回答 0