4

I have a MySQL database "DB" in which there's a table "TABLE" of a single field and it contains about 8 Million rows. I am trying to download that table into a text file as follows:

//Open connection.
connection.Open();

//Create command.
MySqlCommand command = connection.CreateCommand();
command.CommandText = "SELECT * FROM `DB`.`TABLE`";

//Execute command.
MySqlDataReader result = command.ExecuteReader();

//If result isn't null
if (result != null)
{
    //Open stream to write result.
    System.IO.StreamWriter file = new System.IO.StreamWriter(@"C:\Users\StackOverflow\Desktop\ID_List.txt", true);

    //For each line row of result.
    while (result.Read())
    {
        //Write result in a line.
        file.WriteLine(result.GetValue(0).ToString());
    }
}

//Close connection.
connection.Close();

Upon running it starts downloading the data and outputing it to the text file but after only a minute or two it gives:

An unhandled exception of type 'MySql.Data.MySqlClient.MySqlException' occurred in test program.exe

Additional information: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

What Might I change or do differently for it to work? Thanks for any suggestions and answers :)

[EDIT]: I added command.CommandTimeout = 240; and tested three times, every time it downloaded for a couple minutes (about 40MB of data each time) before giving:

An unhandled exception of type 'MySql.Data.MySqlClient.MySqlException' occurred in test program.exe

Additional information: Fatal error encountered during data read.

And these are the last few lines of the debug output:

at MySql.Data.MySqlClient.NativeDriver.FetchDataRow(Int32 statementId, Int32 columns) in :line 0
   at MySql.Data.MySqlClient.Driver.FetchDataRow(Int32 statementId, Int32 columns) in :line 0
   at MySql.Data.MySqlClient.ResultSet.GetNextRow() in :line 0
   at MySql.Data.MySqlClient.ResultSet.NextRow(CommandBehavior behavior) in :line 0
   at MySql.Data.MySqlClient.MySqlDataReader.Read() in :line 0</ExceptionString><InnerException><ExceptionType>System.IO.EndOfStreamException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType><Message>Attempted to read past the end of the stream.</Message><StackTrace>   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count) in :line 0
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket() in :line 0</StackTrace><ExceptionString>System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count) in :line 0
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket() in :line 0</ExceptionString></InnerException></InnerException></Exception></TraceRecord>
An unhandled exception of type 'MySql.Data.MySqlClient.MySqlException' occurred in test program.exe
Additional information: Fatal error encountered during data read.
The program '[4548] test program.exe: Program Trace' has exited with code 0 (0x0).
The program '[4548] test program.exe: Managed (v4.0.30319)' has exited with code 0 (0x0).

[EDIT2]: I added MySqlDataReader result = command.ExecuteReader(System.Data.CommandBehavior.SingleResult); in an effort to remedy the above error and now I am getting:

An unhandled exception of type 'MySql.Data.MySqlClient.MySqlException' occurred in test program.exe
Additional information: Query execution was interrupted

After some light google-ing I figure it is GoDaddy that is limiting the query execution time. Therefore a working solution is to have try/catch with a counter to keep track of how many records were retrieved before the connection was closed and to re-open new connections with the counter as starting point to LIMIT until the table size is exhausted.

Thanks for all the help!

4

3 回答 3

2

增加 my.ini 文件中的 wait_timeout 参数。

http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_wait_timeout

还有其他超时参数可能是问题，但我认为它是一个。

如果您无权访问您的 ini 文件，请使用“限制”来获得有限的结果集。

首先，获取总行数 (count(*) ) 其次，通过 for 获取小的结果集

 for (int i=0; i<=numberOfRowsInYourTable ; i=i+1000){
      command.CommandText = "SELECT * FROM `DB`.`TABLE` LIMIT " + i + " " + new String(i + 1000);
      // Do what you want
 }
于 2013-07-08T12:00:43.347 回答
1

将此添加到您的连接字符串中：

default command timeout=240

并根据需要进行调整。

于 2013-07-08T12:06:12.813 回答
0

你有几个选择

  1. 您可以设置更长的超时时间，这意味着您需要在连接字符串中设置CommandTimeout参数集

  2. 您可以使用SELECT INTO在服务器上生成文件

  3. 您可以使用mysqldump实用程序，该实用程序也会在服务器上生成文件

EDIT Fellow SO 用户提到了其他类型的超时参数。我不是用户，这里的关键是哪一个

于 2013-07-08T12:09:23.640 回答