16

我有一个这样的 .csv 文件数据

Date,Name,Call Type,Number,Duration,Address,PostalCode,City,State,Country,Latitude,Longitude
"Sep-18-2013 01:53:45 PM","Unknown","outgoing call",'123456',"0 Secs","null","null","null","null","null",0.0,0.0,,,
"Sep-18-2013 01:54:14 PM","Unknown","outgoing call",'1234567890',"0 Secs","null","null","null","null","null",0.0,0.0,,,
"Sep-18-2013 01:54:37 PM","Unknown","outgoing call",'14772580369',"1 Secs","null","null","null","null","null",0.0,0.0,,,

我正在使用以下代码将数据插入数据库

$sql = "LOAD DATA INFILE `detection.csv`
              INTO TABLE `calldetections`
              FIELDS TERMINATED BY '".@mysql_escape_string(",").
             "` OPTIONALLY ENCLOSED BY `".@mysql_escape_string("\"").
             "` OPTIONALLY ENCLOSED BY `".@mysql_escape_string("\'").
             "` ESCAPED BY `".@mysql_escape_string("\\").
              "` LINES TERMINATED BY `".",,,\\r\\n".
             "`IGNORE 1 LINES `"

             ."(`date`,`name`,`type`,`number`,`duration`,`addr`,`pin`,`city`,`state`,`country`,`lat`,`log`)";
      $res = @mysql_query($con,$sql);

但没有插入任何内容；错误在哪里？

4

2 回答 2

45

如果您在执行echo($sql);之前这样做，您会发现查询的语法不正确，原因如下：

  1. 文件名应该用引号而不是反引号括起来，因为它是字符串文字而不是标识符。

  2. 绝对不需要调用在and和子句mysql_escape_string()中指定分隔符。FIELDS TERMINATED BYENCLOSED BYESCAPED BY

  3. 你过度使用反引号。实际上，在您的情况下，由于没有使用保留字，因此您将它们全部丢弃。它们只会增加混乱。

  4. 在 CSV 文件的第一行末尾，您必须拥有， ,,,因为您将它们用作行分隔符的一部分。如果您不这样做，您不仅会跳过第一行，还会跳过包含数据的第二行。

  5. 您不能ENCLOSED BY多次使用子句。您必须以不同的方式处理Number字段。

  6. 查看您不需要的示例行恕我直言ESCAPED BY。但是，如果您觉得需要它，请像这样使用它ESCAPED BY '\\'。

话虽如此，一个语法正确的语句可能看起来像这样

LOAD DATA INFILE 'detection.csv'
INTO TABLE calldetections
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"' 
LINES TERMINATED BY ',,,\r\n'
IGNORE 1 LINES 
(date, name, type, number, duration, addr, pin, city, state, country, lat, log)

现在恕我直言，您需要在加载它们时转换相当多的字段：

  1. 如果date你的表是datetime数据类型，那么它需要被转换，否则你会得到一个错误

    不正确的日期时间值：“日期”列的“2013 年 9 月 18 日 01:53:45 PM”

  2. 你必须处理围绕Number字段值的单个 qoutes

  3. 您很可能希望将"null"字符串文字更改NULL为addr, pin, city, state, country列的实际值

  4. 如果持续时间总是以秒为单位，那么您可以提取一个整数秒值并将其存储在您的表中，以便以后能够轻松地聚合持续时间值。

话虽这么说，该声明的有用版本应该看起来像这样

LOAD DATA INFILE 'detection.csv'
INTO TABLE calldetections
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"' 
LINES TERMINATED BY ',,,\r\n'
IGNORE 1 LINES 
(@date, name, type, @number, @duration, @addr, @pin, @city, @state, @country, lat, log)
SET date = STR_TO_DATE(@date, '%b-%d-%Y %h:%i:%s %p'),
    number = TRIM(BOTH '\'' FROM @number),
    duration = 1 * TRIM(TRAILING 'Secs' FROM @duration),
    addr = NULLIF(@addr, 'null'),
    pin  = NULLIF(@pin, 'null'),
    city = NULLIF(@city, 'null'),
    state = NULLIF(@state, 'null'),
    country = NULLIF(@country, 'null')

下面是在我的机器上执行查询的结果

mysql> 加载数据输入文件'/tmp/detection.csv'
    -> INTO TABLE 调用检测
    -> 由“，”终止的字段
    -> 可选地由 '"' 包围
    -> 以 ',,,\n' 结尾的行
    -> 忽略 1 行
    ->（@日期、名称、类型、@number、@duration、@addr、@pin、@city、@state、@country、lat、log）
    -> 设置日期 = STR_TO_DATE(@date, '%b-%d-%Y %h:%i:%s %p'),
    -> number = TRIM(BOTH '\'' FROM @number),
    -> 持续时间 = 1 * TRIM(TRAILING 'Secs' FROM @duration),
    -> addr = NULLIF(@addr, 'null'),
    -> pin = NULLIF(@pin, 'null'),
    -> 城市 = NULLIF(@city, 'null'),
    -> 状态 = NULLIF(@state, 'null'),
    -> 国家 = NULLIF(@country, 'null');
查询正常，3 行受影响（0.00 秒）
记录：3 删除：0 跳过：0 警告：0

mysql> select * from calldetections;
+----------+---------+--------------+- ------------+----------+------+------+------+----- --+---------+------+------+
| 日期 | 姓名 | 类型 | 号码 | 持续时间 | 地址 | 销 | 城市| 状态 | 国家 | 纬度 | 日志 |
+----------+---------+--------------+- ------------+----------+------+------+------+----- --+---------+------+------+
| 2013-09-18 13:53:45 | 未知 | 拨出电话 | 123456 | 0 | 空 | 空 | 空 | 空 | 空 | 0.0 | 0.0 |
| 2013-09-18 13:54:14 | 未知 | 拨出电话 | 1234567890 | 0 | 空 | 空 | 空 | 空 | 空 | 0.0 | 0.0 |
| 2013-09-18 13:54:37 | 未知 | 拨出电话 | 14772580369 | 1 | 空 | 空 | 空 | 空 | 空 | 0.0 | 0.0 |
+----------+---------+--------------+- ------------+----------+------+------+------+----- --+---------+------+------+
3 行一组（0.00 秒）

最后在 php 中将查询字符串分配给$sql变量应该如下所示

$sql = "LOAD DATA INFILE 'detection.csv'
        INTO TABLE calldetections
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY ',,,\\r\\n'
        IGNORE 1 LINES 
        (@date, name, type, @number, @duration, @addr, @pin, @city, @state, @country, lat, log)
        SET date = STR_TO_DATE(@date, '%b-%d-%Y %h:%i:%s %p'),
            number = TRIM(BOTH '\'' FROM @number),
            duration = 1 * TRIM(TRAILING 'Secs' FROM @duration),
            addr = NULLIF(@addr, 'null'),
            pin  = NULLIF(@pin, 'null'),
            city = NULLIF(@city, 'null'),
            state = NULLIF(@state, 'null'),
            country = NULLIF(@country, 'null') ";
于 2013-09-22T07:29:07.260 回答
1

1分钟内在数据库中批量插入超过7000000条记录（带计算的超快速查询）

    mysqli_query($cons, '
    LOAD DATA LOCAL INFILE "'.$file.'"
    INTO TABLE tablename
    FIELDS TERMINATED by \',\'
    LINES TERMINATED BY \'\n\'
    IGNORE 1 LINES
    (isbn10,isbn13,price,discount,free_stock,report,report_date)
     SET RRP = IF(discount = 0.00,price-price * 45/100,IF(discount = 0.01,price,IF(discount != 0.00,price-price * discount/100,@RRP))),
         RRP_nl = RRP * 1.44 + 8,
         RRP_bl = RRP * 1.44 + 8,
         ID = NULL
    ')or die(mysqli_error());
    $affected = (int) (mysqli_affected_rows($cons))-1; 
    $log->lwrite('Inventory.CSV to database:'. $affected.' record inserted successfully.');

RRP 和 RRP_nl 和 RRP_bl 不在 csv 中，但我们是在插入之后计算出来的。

于 2017-02-17T07:56:40.527 回答