0

我正在从固定长度的平面文件中读取数据，并应用了以下脚本：

CREATE EXTERNAL TABLE `test_table`.`test_data`
(test_column1 STRING,
test_column2 STRING
) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' 
WITH SERDEPROPERTIES ("input.regex" = "(.{10})(.{10})" )
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION '<specified location>';

我运行了select * from `test_table`.`test_data` 查询，它运行良好。

当我运行select count(1) from `test_table`.`test_data`查询时，它给出了以下错误。

SQL Error [2] [08S01]: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1620900284233_0004_195_00, diagnostics=[Task failed, taskId=task_1620900284233_0004_195_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1620900284233_0004_195_00_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Error creating SerDe for LLAP IO

谁能告诉我究竟Error creating SerDe for LLAP IO意味着什么，以及我该如何解决这个问题？

4

1 回答 1

1

问题解决了。以下答案引用自此处

'RegexSerDe' is part of 'hive-contrib' library. From Hive 0.10 onwards the serde is part of 'hive-serde-<version>.jar'

Create the table with latest serde 'org.apache.hadoop.hive.serde2.RegexSerDe' instead of 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'

You can also run alter table to modify the serde as below

ALTER TABLE <TABLENAME> SET SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe';
于 2021-05-18T06:15:38.543 回答