我正在使用下表来处理大约 15GB（.gz 压缩）的 iislog。使用亚马逊 EMR（1 个中型主实例、4 个大型核心实例、2 个任务实例）。甚至需要大约 1 小时才能获得此查询的结果：
select uri, cs_Cookie as Cookie, count(*) as hits from tmp1 group by cs_Cookie, uri order by hits Desc;
我看到所有 DataNode 上的 cpu 利用率每次都是 100%。那么，有人可以建议如何减少查询时间和cpu利用率吗？
表定义：
create external table marData(logdate string, time string, computername string, clientip string, uri string, qs string, localfile string, status string, referer string, w3status string, sc_bytes string, cs_bytes string, w3wpbytes string, cs_username string, cs_user_agent string, time_local string, timetakenms string, sc_substatus string, s_sitename string, s_ip string, s_port string, RequestsPerSecond string, s_proxy string, cs_version string, c_protocol string, cs_method string, cs_Host string, EndRequest_UTC string, date_local string, CPU_Utilization string, cs_Cookie string, BeginRequest_UTC string) partitioned by (month string) ROW FORMAT SERDE
'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
"input.regex" ="([0-9-]+) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\".*\"|[^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\".*\"|[^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\".*\"|[^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([0-9-]+ [0-9:.]+) ([^ ]*) ([^ ]*) (\".*\"|[^ ]*) ([0-9-]+ [0-9:.]+)",
"output.format.string"="%1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s %11$s %12$s %13$s %14$s %15$s %16$s %17$s %18$s %19$s %20$s %21$s %22$s %23$s %24$s %25$s %26$s %27$s %28$s %29$s %30$s %31$s %32$s")
location 's3://logdata/Mar';