c# - 提高 SQLite 性能

Question

好吧，我有一个file.sql包含20,000插入命令的

.sql来自文件的样本

插入table值（1,-400,400,3,154850,'Text',590628,'TEXT',1610,'TEXT',79）；

插入table值（39,-362,400,3,111659,'Text',74896,'TEXT',0,'TEXT',14）；

我正在使用以下代码创建内存中的 Sqlite 数据库并将值拉入其中然后计算经过的时间

using (var conn = new SQLiteConnection(@"Data Source=:memory:"))
{
    conn.Open();

    var stopwatch = new Stopwatch();
    stopwatch.Start();

    using (var cmd = new SQLiteCommand(conn))
    {
        using (var transaction = conn.BeginTransaction())
        {

                cmd.CommandText = File.ReadAllText(@"file.sql");
                cmd.ExecuteNonQuery();

            transaction.Commit();
        }
    }

    var timeelapsed = stopwatch.Elapsed.TotalSeconds <= 60
                          ? stopwatch.Elapsed.TotalSeconds + " seconds"
                          : Math.Round(stopwatch.Elapsed.TotalSeconds/60) + " minutes";
    MessageBox.Show(string.Format("Time elapsed {0}", timeelapsed));
    conn.Close();
}

我尝试过的事情

1. 使用文件数据库而不是内存之一。
2. 使用开始事务和提交事务 [如我的代码中所示]。
3. Using Firefox's extension named SQLite Manager to test whether the slowing down problem is from the script; However, I was surprised that the same 20,000 lines that i am trying to process using my code has been pulled to the database in JUST 4ms!!!.
4. Using PRAGMA synchronous = OFF, as well as, PRAGMA journal_mode = MEMORY.
5. Appending begin transaction; and commit transaction; to the beginning and ending of the .sql file respectively.

As the SQLite documentations says : SQLite is capable of processing 50,000 commands per seconds. And that is real and i made sure of it using the SQLite Manager [AS DESCRIPED IN THE THIRD SOMETHING THAT I'V TRIED]; However, I am getting my 20,000 commands done in 4 minutes something that tells that there is something wrong.

QUESTION : What is the problem am i facing why is the Execution done very slowly ?!

Answer 1

SQLite.Net documentation recommends the following construct for transactions

using (SqliteConnection conn = new SqliteConnection(@"Data Source=:memory:")) 
{
  conn.Open();
  using(SqliteTransaction trans = conn.BeginTransaction())
  {
    using (SqliteCommand cmd = new SQLiteCommand(conn))
    {
      cmd.CommandText = File.ReadAllText(@"file.sql");
      cmd.ExecuteNonQuery();
    }
    trans.Commit();
  }
  con.Close();
}

Answer 2

Are you able to manipulate the text file contexts to something like:

INSERT INTO table (col01, col02, col03, col04, col05, col06, col07, col08, col09, col10, col11)
SELECT 1,-400,400,3,154850,'Text',590628,'TEXT',1610,'TEXT',79
UNION ALL
SELECT 39,-362,400,3,111659,'Text',74896,'TEXT',0,'TEXT',14
;

Maybe try "batching them" into groups of 100 as a initial test.

http://sqlite.org/lang_select.html

SqlLite seems to support the UNION ALL statement.