c# - Ways to increase performance of DataTable.Load()?

Question

I currently use a custom CSV class from Codeproject to create a CSV object. I then use this to populate a DataTable. Under profiling this is taking more time than I would like and I wonder if there is a more efficient way of doing it?

The CSV contains approximately 2,500 rows and 500 columns.

The CSV reader is from: http://www.codeproject.com/Articles/9258/A-Fast-CSV-Reader

StreamReader s = new StreamReader(confirmedFilePath);
CsvReader csv = new CsvReader(s, true);
DataTable dt = new DataTable();
dt.Load(csv);

I came across a google search suggesting a DataAdapter, but it was only one reference to this? I searched further but didn't find any collaboration.

Answer 1

CsvReader 快速且可靠，我几乎可以肯定您找不到更快的（如果有的话）来读取 CSV 数据。

限制来自处理新数据的DataTable，2500 * 500就够了。我认为最快的方法是直接 CsvReader->DataBase (ADO.NET) 链。

Answer 2

试试GenericParser 。

Answer 3

从数据库填充时始终使用BeginLoadData()and EndLoadData()，因为它们已经自己强制执行约束 - 唯一的缺点是 CSV 文件显然没有，因此只有在整个操作结束后才会引发任何异常。

...
dt.BeginLoadData();
dt.Load(csv, LoadOption.Upsert);
dt.EndLoadData();

编辑：LoadOption.Upsert仅当DataBase为空，或者您不想保留对现有数据的任何先前更改时才使用- 这样更快。