2

I'm building a dynamic SQL statement, that will insert one or more sets of VALUES via a prepared DBI statement, my question is this:

Since I have a dynamic number of VALUES sets, and I will need to add as many ( ?, ?, ?),( ?, ?, ?) etc as necessary to extend the statement INSERT INTO `tblname` ( $columnsString ) VALUES in order to submit only one query using placeholders and bind values- is this the preferred method(most efficient, etc., - reasoning behind efficiency would be helpful in your answer if possible) or should I just be building this as a query string with sprintf and dbh->quote()?

(As a little extra information: I'm actually using AnyEvent::DBI right now, which only exposes placeholders & bind values and not the quote() method so this wouldn't be easy for me to accomplish without creating another straight DBI $dbh and using another db server connection just to use the quote() method, or without altering the AnyEvent::DBI module myself.)

Normally I would just execute the statements as necessary but in this heavy workload case I'm trying to batch inserts together for some DB efficiency.

Also, if anyone could answer if it is possible( and then how to ) insert an sql DEFAULT value using placeholders and bind values that'd be awesome. Typically if I ever needed to do that I'd append the DEFAULTs to the string directly and use sprintf and $dbh->quote() only for the non DEFAULT values.

UPDATE:

Worked out the misunderstanding in a quick chat. User ikegami suggested that instead of building the query string myself without placeholders, that I just intermingle VALUES and placeholders such as:

$queryString .= '(DEFAULT,?,?),(DEFAULT,DEFAULT,DEFAULT)';

Some of the reasoning behind my first asking of this question on SO was because I was somewhat against this intermingling due to my thought that it made the code less readable, though after being assured that sql 'DEFAULT' couldn't be in a placeholder bind value, this was the method I had begun implementing.

Using placeholders where possible does seem to be the more accepted method of building queries, and if you want an SQL DEFAULT you just need to include it in the same query building as the placeholders. This does not apply to NULL values, as those CAN be inserted with placeholders and a bind value of undef.

Update 2:

The reasoning I asked about performance, the 'acceptance' of building your own query with quote() vs building with placeholders, and why I've gone with a solution that involves using all columns for the SQL INSERT INTO tblname (cols) is because I have roughly 2-4 million rows a day going into a terrible db server, and my code is running on an equally terrible server. With my requirements of needing DEFAULT sql values, and these terrible performance constraints, I've chosen a solution for now.

For future devs who stumble upon this - take a look at @emazep's solution of using SQL::Abstract, or if for some reason you need to build your own, you might consider either using @Schwern's subroutine solution or possibly incorporating some of @ikegami's answer into it as these are all great answers as to the 'Current state of affairs' regarding the usage of DBI and building dynamic queries.

4

4 回答 4

3

除非有特定的理由重新发明轮子（可能有一些），SQL::Abstract（其中）已经为我们所有人解决了动态 SQL 生成的问题：

my %data = (
    name    => 'Jimbo Bobson',
    phone   => '123-456-7890',
    address => '42 Sister Lane',
    city    => 'St. Louis',
    state   => 'Louisiana'
);

use SQL::Abstract;
my ($stmt, @bind)
    = SQL::Abstract->new->insert('people', \%data);

print $stmt, "\n";
print join ', ', @bind;

打印：

INSERT INTO people ( address, city, name, phone, state)
VALUES ( ?, ?, ?, ?, ? )
42 Sister Lane, St. Louis, Jimbo Bobson, 123-456-7890, Louisiana

SQL::Abstract然后提供了一个很好的技巧来迭代许多行以插入，而无需每次都重新生成 SQL，但是对于批量插入，还有SQL::Abstract::Plugin::InsertMulti

use SQL::Abstract;    
use SQL::Abstract::Plugin::InsertMulti;

my ($stmt, @bind)
    = SQL::Abstract->new->insert_multi( 'people', [
        { name => 'foo', age => 23 },
        { name => 'bar', age => 40 },
    ]);

# INSERT INTO people ( age, name ) VALUES ( ?, ? ), ( ?, ? )
# 23, foo, 40, bar
于 2012-10-05T00:39:29.450 回答
2

我有时会使用如下构造：

#!/usr/bin/env perl

use strict; use warnings;

# ...

my @columns = ('a' .. 'z');

my $sql = sprintf(q{INSERT INTO sometable (%s) VALUES (%s)},
    join(',', map $dbh->quote($_), @columns),
    join(',', ('?') x @columns),
);

至于处理DEFAULT，是否会保留该列以确保数据库将其设置为默认值？

于 2012-10-04T16:57:36.067 回答
1

如果您将占位符用于“静态”查询，那么您也应该将它们用于“动态”查询。查询就是查询。

my $stmt = 'UPDATE Widget SET foo=?'
my @params = $foo;

if ($set_far) {
   $stmt .= ', far=?';
   push @params, $far;
}

{
   my @where;

   if ($check_boo) {
      push @where, 'boo=?';
      push @params, $boo;
   }

   if ($check_bar) {
      push @where, 'bar=?';
      push @params, $bar;
   }

   $stmt .= ' WHERE ' . join ' AND ', map "($_)", @where
      if @where;
}

$dbh->do($stmt, undef, @params);

我使用了 UPDATE，因为它允许我展示更多内容，但所有内容也适用于 INSERT。

my @fields = ('foo');
my @params = ($foo);

if ($set_far) {
   push @fields, 'bar';
   push @params, $far;
}

$stmt = 'INSERT INTO Widget ('
      . join(',', @fields)
      . ') VALUES ('
      . join(',', ('?')x@fields)
      . ')';

$dbh->do($stmt, undef, @params);
于 2012-10-04T16:51:42.603 回答
1

您已经表达了对代码可读性以及能够传入 DEFAULT 的担忧。我会把@ikegami 的回答更进一步......

sub insert {
    my($dbh, $table, $fields, $values) = @_;

    my $q_table      = $dbh->quote($table);
    my @q_fields     = map { $dbh->quote($_) } @$fields;
    my @placeholders = map { "?" } @q_fields;

    my $sql = qq{
        INSERT INTO $q_table
               ( @{[ join(', ', @q_fields)    ]} )
        VALUES ( @{[ join(', ', @placeholders ]} )
    };

    return $dbh->do($sql, undef, @$values);
}

现在您有了一个通用的多值插入例程。

# INSERT INTO foo ('bar', 'baz') VALUES ( 23, 42 )
insert( $dbh, "foo", ['bar', 'baz'], [23, 43] );

要指示默认值，请不要传入该列。

# INSERT INTO foo ('bar') VALUES ( 23 )
# 'baz' will use its default
insert( $dbh, "foo", ['bar'], [23] );

您可以对此进行优化，以使您的子例程通过一个子例程调用和一个准备好的语句执行多次插入，从而在客户端节省 CPU（如果它支持准备好的句柄，则可能在数据库端进行一些操作）。

sub insert {
    my($dbh, $table, $fields, @rows) = @_;

    my $q_table      = $dbh->quote($table);
    my @q_fields     = map { $dbh->quote($_) } @$fields;
    my @placeholders = map { "?" } @q_fields;

    my $sql = qq{
        INSERT INTO $q_table
               ( @{[ join(', ', @q_fields)    ]} )
        VALUES ( @{[ join(', ', @placeholders ]} )
    };

    my $sth = $dbh->prepare_cached($sql);
    for my $values (@rows) {
        $sth->execute(@$values);
    }
}

# INSERT INTO foo ('bar', 'baz') VALUES ( 23, 42 )
# INSERT INTO foo ('bar', 'baz') VALUES ( 99, 12 )
insert( $dbh, "foo", ['bar', 'baz'], [23, 43], [99, 12] );

最后，您可以编写一个批量插入，在单个语句中传递多个值。这可能是进行大量插入的最有效方法。这就是拥有一组固定列并传入DEFAULT标记的地方。我使用了作为标量引用传递的值被视为原始 SQL 值的习语。现在你可以灵活地传递你喜欢的任何东西。

sub insert {
    my($dbh, $table, $fields, @rows) = @_;

    my $q_table      = $dbh->quote($table);
    my @q_fields     = map { $dbh->quote($_) } @$fields;

    my $sql = qq{
        INSERT INTO $q_table
               ( @{[ join(', ', @q_fields)    ]} )
        VALUES
    };

    # This would be more elegant building an array and then joining it together
    # on ",\n", but that would double the memory usage and there might be
    # a lot of values.
    for my $values (@rows) {
        $sql .= "( ";
        # Scalar refs are treated as bare SQL.
        $sql .= join ", ", map { ref $value ? $$_ : $dbh->quote($_) } @$values;
        $sql .= "),\n";
    }
    $sql =~ s{,\n$}{};

    return $dbh->do($sql);
}

# INSERT INTO foo ('bar', 'baz') VALUES ( 23, NOW ), ( DEFAULT, 12 )
insert( $dbh, "foo", ['bar', 'baz'], [23, \"NOW"], [\"DEFAULT", 12] );

不利的一面是这会在内存中构建一个字符串，可能非常大。要解决这个问题，您必须从文件语法中涉及特定于数据库的批量插入。

与其自己编写所有这些 SQL 生成内容，不如使用@emazep 的答案并使用 SQL::Abstract 和SQL::Abstract::Plugin::InsertMulti。

只要确保你的个人资料。

于 2012-10-04T20:28:37.923 回答