0

我正在尝试从我的 .NET 应用程序运行 SQL 文件,但运行时间非常慢。我不确定如何优化它以更快地运行。我知道该脚本有效,因为当我在 SQLite shell 应用程序中运行它时,它会在不到 1 秒的时间内执行。在我的 .NET 应用程序中,它只是挂在sqlite_cmd.ExecuteNonQuery().

以下是 script.sql 文件的摘录:

--
-- File generated with SQLiteStudio v3.1.1 on Thu Feb 15 11:33:12 2018
--
-- Text encoding used: System
--
PRAGMA foreign_keys = off;
BEGIN TRANSACTION;

-- Table: Document_Type
CREATE TABLE [Document_Type] (
    [Id]    integer PRIMARY KEY AUTOINCREMENT NOT NULL,
    [Type]  nvarchar(50) NOT NULL COLLATE NOCASE

);
INSERT INTO Document_Type (Id, Type) VALUES (1, '.docx');
INSERT INTO Document_Type (Id, Type) VALUES (2, '.xlsm');
INSERT INTO Document_Type (Id, Type) VALUES (3, '.jpeg');

-- Table: Learn_More
CREATE TABLE [Learn_More] (
    [item_ID]   integer PRIMARY KEY AUTOINCREMENT NOT NULL,
    [field_name]    nvarchar(100) COLLATE NOCASE,
    [description]   text(1073741823) COLLATE NOCASE,
    [location]  char(10) COLLATE NOCASE

);
INSERT INTO Learn_More (item_ID, field_name, description, location) VALUES (1, 'System Name', 'Full descriptive name of the system.*      Example: Agency Billing System ', 'SSP1      ');
.
.
.
.
.
COMMIT TRANSACTION;
PRAGMA foreign_keys = on;

script.sql 文件有 35,831 行。我计划在可以在 .NET 中运行之后清理列类型。

我的 .NET 代码:

Dim baseDir As String = Application.UserAppDataPath()
Dim db3Path As String = baseDir + "\Schema Scripts\ProgrammaticallyGenerateDatabase\Test.db3"
Dim sqlPath As String = baseDir + "\Schema Scripts\script.sql"

SQLiteConnection.CreateFile(db3Path)

Dim sqlite_conn = New SQLiteConnection("Data Source= " & db3Path)
Using sqlite_conn
    sqlite_conn.Open()
    Dim sqlite_cmd = sqlite_conn.CreateCommand()
    'Dim sqlite_tran = sqlite_conn.BeginTransaction()

    'Using sqlite_tran

        Using sqlite_cmd
            Try
                sqlite_cmd.CommandText = File.ReadAllText(sqlPath)
                sqlite_cmd.ExecuteNonQuery()

                sqlite_cmd.CommandText = "SELECT Type FROM Document_Type"

                Dim sqlite_datareader = sqlite_cmd.ExecuteReader()
                While (sqlite_datareader.Read())
                    Dim textReader As String = sqlite_datareader.GetString(0)

                    Console.WriteLine(textReader)
                End While
            Catch ex As Exception
                'End Using
                'sqlite_tran.Commit()
            End Try
        End Using
    sqlite_conn.Close()
End Using

我试过使用File.OpenText(sqlPath).ReadToEnd()但有同样的性能问题。

关于如何加快速度的任何想法?.read script.sql重申一下,使用该命令在 SQLite shell 程序中运行相同的文件不到 1 秒。

编辑:

这是一个获胜的应用程序。代码在后台线程上运行,需要 483801 毫秒才能完成。

我想出了一个适合我的情况的解决方案。查询现在在 ~700-900 毫秒内运行。我只是将 sqlite3.exe 与我的应用程序捆绑在一起并使用命令行调用它。

这是代码:

Private Sub SQLiteButtonPress()
        Dim baseDir As String = Application.UserAppDataPath() + "\Schema Scripts"
        Dim db3Path As String = baseDir + "\ProgrammaticallyGenerateDatabase\Test.db3"
        Dim sqlPath As String = baseDir + "\Baseline Schema.sql"

        Dim watch = System.Diagnostics.Stopwatch.StartNew()
        Console.WriteLine("Starting Stopwatch")

        Try
            Dim proc As New System.Diagnostics.ProcessStartInfo()
            proc.Arguments = "/c sqlite3.exe """ + db3Path + """ < """ + sqlPath + """"
            proc.FileName = "cmd.exe"
            proc.UseShellExecute = False
            proc.CreateNoWindow = True

            Dim p As New Process()
            p.StartInfo = proc
            p.Start()
            p.WaitForExit()

            Dim sqlite_conn = New SQLiteConnection("Data Source= " & db3Path)
            Using sqlite_conn
                sqlite_conn.Open()
                Dim sqlite_cmd = sqlite_conn.CreateCommand()

                Using sqlite_cmd
                        sqlite_cmd.CommandText = "SELECT Type FROM Document_Type"

                        Dim sqlite_datareader = sqlite_cmd.ExecuteReader()
                        While (sqlite_datareader.Read())
                            Dim textReader As String = sqlite_datareader.GetString(0)

                            Console.WriteLine(textReader)
                        End While
                End Using
                sqlite_conn.Close()
            End Using

        Catch ex As Exception
            Throw ex
        End Try

        watch.Stop()
        Dim elapsedMs = watch.ElapsedMilliseconds
        Console.WriteLine("Finished Stopwatch: " & elapsedMs & "ms")
    End Sub
4

2 回答 2

0

出于某种原因,.NET SQLite 库不会很快在单个字符串中运行大型查询,即使您将每个批量插入包装到事务中也是如此;或将整个“ExecuteNonQuery”方法调用包装在事务中。

上述解决方案有效,将您的大型 SQL 文件拆分为单独的语句,在事务中执行所有语句,然后提交。通过这种方式,我能够在 0.22 秒内将 15,000 条记录加载到基于内存的 SQLite 实例中。

干杯。

于 2019-03-21T12:24:47.310 回答
0

我有同样的问题。@MarkBenningfield 的建议解决了这个问题。正如他所提到的,将您的脚本拆分为分号,并在循环中分别运行每个语句。在那之后,一切对我来说都运行得非常快。

于 2019-01-18T08:42:32.740 回答