这是我第一次使用 Polybase,我正在尝试将示例 CSV 文件(以第一条记录作为标题)从ADLS Gen2加载到 Synapse。我之前已经创建了一个万能钥匙,所以我没有再次创建它。我实施的其余步骤如下:
-- Step 1
CREATE DATABASE SCOPED CREDENTIAL access_cred
WITH
IDENTITY = 'my_name',
SECRET = '12345678910****==';
-- Step 2
CREATE EXTERNAL DATA SOURCE CreditCards
WITH
(
TYPE = HADOOP,
LOCATION = 'abfss://container01@freesandbox.dfs.core.windows.net',
CREDENTIAL = access_cred
);
-- Step 3
CREATE EXTERNAL FILE FORMAT CC_FileFormat
WITH
(
FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS
(
FIELD_TERMINATOR = ',',
FIRST_ROW = 2,
USE_TYPE_DEFAULT = FALSE
)
);
--Step 4
CREATE SCHEMA ext;
CREATE SCHEMA cc;
--Step 5: Create External Table
CREATE EXTERNAL TABLE ext.creditcards (
Card_Type_Full_Name varchar(50),
Issuing_Bank varchar(50),
Card_Number varchar(50),
Card_Holder_Name varchar(50),
CVV_CVV2 varchar(50),
Issue_Date varchar(50),
Expiry_Date varchar(50),
Billing_Date varchar(50),
Card_PIN varchar(50),
Credit_Limit varchar(50)
)
WITH (LOCATION='/CreditCards/', --I've a folder 'CreditCards' inside which the 'Creditcards.csv` file sits
DATA_SOURCE = CreditCards,
FILE_FORMAT = CC_FileFormat,
REJECT_TYPE = VALUE,
REJECT_VALUE = 0
);
--Step 6
CREATE TABLE cc.creditcards
WITH
(
DISTRIBUTION = REPLICATE,
CLUSTERED COLUMNSTORE INDEX
)
AS
SELECT * FROM ext.creditcards
OPTION (LABEL = 'CTAS : Load cc.creditcards');
我不知道我做错了什么。我尝试通过互联网浏览多个帖子,但没有真正解决我面临的问题。尝试执行以下操作时出现以下错误SELECT * from <external_table>
。(这也是我在第 6 步中遇到的错误,因为我是 CTAS 命令来加载我的决赛桌。)
Msg 107090, Level 16, State 1, Line 74
HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 1
我不明白还缺少什么。我也授予了容器级别的访问权限。有人可以帮我解决这个问题吗?