0

我有一个来自 SSMS 的 .rpt 文件输出。我可以在文本编辑器或 Excel 中打开它，但格式似乎不是空格、制表符或逗号分隔，因此导致 Excel 中的格式不正确。

我在使用LOAD DATA INFILE调用加载到 MySQL 时遇到了类似的问题。

即，加载“完成”，但由于明显的格式差异，大多数行/列被跳过。

输入：

EffectiveDate           family Instrument  SedolCode    Name                                                         ICB         QZ                     VZ                     MZ                     SZ                     volZ                   LZ                     DYZ                    QS                     VS                     MS                     SS                     volS                   LS                     DYS                    price_o                fx_o                   shares                 cap_o_usd
----------------------- ------ ----------- ------------ ------------------------------------------------------------ ----------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ---------------------- ----------------------
1998-03-23 00:00:00.000 RU1000 1010846     2046789      Arco Chemical                                                1           0.332041753946526      3                      -0.701907649892818     0                      1.12824639032619       1.25544514744965       1.60431785387846       0.630071248806063      0.998650187313827      0.241368465365155      0.5                    0.870392090484651      0.895341500804304      0.945678041846726      48.56                  1                      16794000               815555357.27
1998-03-23 00:00:00.000 RU1000 1004415     2048804      Pinnacle West Capital Corp                                   7           0.197368136475134      0.295748219600254      0.0886833678439797     0                      1.05597173533583       -0.0327467636082814    0.75373010240488       0.578230241112579      0.616288903631446      0.535333094702153      0.5                    0.854509410778738      0.486938352167234      0.774494234589535      44.44                  1                      84718000               3764645032.54
1998-03-23 00:00:00.000 RU1000 1003705     2050832      Ameren Corp                                                  7           0.482269760349473      -0.0732144149931671    -0.623013812458784     0                      1.29932391788085       -0.218587562254685     1.72709462374024       0.685192934274198      0.470817875898523      0.266637746506796      0.5                    0.903083725664561      0.413485670414547      0.9579246320922        42.31                  1                      137215000              5805918920.82

表格格式：

create table myTable(
    effectiveDate DATE NOT NULL,
    family CHAR(6) NOT NULL,
    instrument INT NOT NULL,
    sedol CHAR(8),
    name VARCHAR(100),
    icb INT,
    QZ DOUBLE,
    VZ DOUBLE,
    MZ DOUBLE,
    SZ DOUBLE,
    volZ DOUBLE,
    LZ DOUBLE,
    DYZ DOUBLE,
    QS DOUBLE,
    VS DOUBLE,
    MS DOUBLE,
    SS DOUBLE,
    volS DOUBLE,
    LS DOUBLE,
    DYS DOUBLE,
    priceO DOUBLE,
    fxO DOUBLE,
    capOUSD DOUBLE,
    PRIMARY KEY (effectiveDate, instrument));

加载调用

load data infile '/ru/z1.rpt'
into table myTable
(effectiveDate, family, instrument, sedol, name, icb, qz, vz, mz, sz, volz, lz, dyz, qs, vs, ms, ss, vols, dys, priceO, fxO, @dummy, capOUSD);
4

2 回答 2

1

您必须加载整个数据行，然后使用适当的数据类型转换对其进行解析：

LOAD DATA INFILE 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\z1.rpt'
INTO TABLE myTable
IGNORE 2 ROWS /* skip header */
(@tmp) /* load whole line into vartiable */
SET  /* then parse separate field values from the variable */
EffectiveDate = SUBSTRING(@tmp,1,23), /* datetime literal - get as-is */
family        = SUBSTRING(@tmp,25,6), /* string data - get needed length */
Instrument    = 0+SUBSTRING(@tmp,32,11), /* numeric data - convert from string implicitly */
Sedol         = SUBSTRING(@tmp,44,8),
Name          = SUBSTRING(@tmp,57,60),
ICB           = 0+SUBSTRING(@tmp,118,12),
QZ            = 0+SUBSTRING(@tmp,130,22),
VZ            = 0+SUBSTRING(@tmp,153,22),
MZ            = 0+SUBSTRING(@tmp,176,22),
SZ            = 0+SUBSTRING(@tmp,199,22),
volZ          = 0+SUBSTRING(@tmp,222,22),
LZ            = 0+SUBSTRING(@tmp,245,22),
DYZ           = 0+SUBSTRING(@tmp,268,22),
QS            = 0+SUBSTRING(@tmp,291,22),
VS            = 0+SUBSTRING(@tmp,314,22),
MS            = 0+SUBSTRING(@tmp,337,22),
SS            = 0+SUBSTRING(@tmp,360,22),
volS          = 0+SUBSTRING(@tmp,383,22),
LS            = 0+SUBSTRING(@tmp,406,22),
DYS           = 0+SUBSTRING(@tmp,429,22),
priceO        = 0+SUBSTRING(@tmp,452,22),
fxO           = 0+SUBSTRING(@tmp,475,22),
/* shares=0+SUBSTRING(@tmp,498,22), */ /* field is absent in table structure - skipped */
capOUSD       = 0+SUBSTRING(@tmp,521,22)
;

  1. 源数据包含shares表结构中不存在的字段（已注释）；

  2. name在表结构中定义，VARCHAR(100)而它的值在数据文件中只有 60 个符号。

  3. 每个文件可能有自己的字段长度 - 因此在导入之前检查这一点（如果需要，请更正）。

于 2020-03-02T09:11:20.630 回答
0

如果你在 unix/linux 上，那么你可以通过 sed 去掉空格。解决方案在这里

我使用PHP为自己找到了一个解决方案：

<?php

$mysqli  =  new mysqli(
"***",
"***",
"***",
"***",
3306
);
mysqli_options($mysqli, MYSQLI_OPT_LOCAL_INFILE, true);

if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
    exit();
}

function createTempFileWithDelimiter($filename, $path){
    $content = file_get_contents($filename);
    $replaceContent = preg_replace('/\ +/', ',', $content); 
    
    $onlyFileName = explode('\\',$filename);

    $newFileName = $path.end($onlyFileName);
    file_put_contents($newFileName, $replaceContent);
    
    return $newFileName;
}

$pathTemp = 'C:\\Temp\\';

$pathToFile = 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\z1.rpt';

$file = createFileWithDelimiter($pathToFile, $pathTemp);
$file = str_replace(DIRECTORY_SEPARATOR, '/', $file);

$sql = "LOAD DATA INFILE '".$file."' INTO TABLE `myTable` 
    COLUMNS TERMINATED BY ','
    LINES TERMINATED BY '\n'
    IGNORE 2 LINES
    (effectiveDate, family, instrument, sedol, name, icb, qz, vz, mz, sz, volz, lz, dyz, qs, vs, ms, ss, vols, dys, priceO, fxO, @dummy, capOUSD);";

if (!($stmt = $mysqli->query($sql))) {
    echo "\nQuery execute failed: ERRNO: (" . $mysqli->errno . ") " . $mysqli->error;
};

unlink($file);
?>

不要在preg_replace中使用 '/\s+/'因为 \s 匹配任何空白字符（相当于 [\r\n\t\f\v ]）并且格式会改变，列和换行符会消失。

于 2021-11-17T01:00:59.677 回答