hadoop - 将文本文件放入 hive 数据库

Question

我试图运行这段代码这么长时间有人能告诉我它的代码有什么问题吗：-

CREATE EXTERNAL TABLE samp_log 
(
ip String ,col1 String ,col2 String , date String , time_hour int ,time_min int 
,time_sec int ,zone int , request String , request_con String , resp_code int 
,resp_byte BIGINT , reference String , ext_reference String , col13 String 
,col14 String ,col15 String , col16 String ,col17 String  
) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' 
WITH SERDEPROPERTIES ("field.delim"=" ,[,]") 
STORED AS TEXTFILE

错误 - 驱动程序返回：1。错误：OK FAILED：执行错误，从 org.apache.hadoop.hive.ql.exec.DDLTask 返回代码 1。无法验证 serde：org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe

我还添加了 hive-contrib 的 jar 文件。

Answer 1

use RegexSerDe

https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ApacheWeblogData

Here is a POC:

create external table mytable  
( 
    ip      string
   ,dt      string
   ,tm      string
   ,tz      string
)
    row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
    with serdeproperties  
    (
        'input.regex' = '^(.*?) - - \\[(.*?):(.*?) (.*?)\\].*$'
    )
    location '/tmp/mytable'
;

---

select * from mytable
;

---

+-----------------+-------------+------------+------------+
|   mytable.ip    | mytable.dt  | mytable.tm | mytable.tz |
+-----------------+-------------+------------+------------+
| 123.123.123.123 | 26/Apr/2000 | 00:23:48   |      -0400 |
+-----------------+-------------+------------+------------+