0

在尝试通过 stringwriter 将数据表数据转换为 xml 时。它给出了内存不足的异常。

我的数据表包含大约 20000 条记录。

下面是我正在尝试的代码

DataTable dt = GetData();// contains 20000 records.

    StringWriter sw = new StringWriter();
    dt.WriteXml(sw);

    XmlDocument sd = new XmlDocument();
    sd.LoadXml(sw.ToString());

有人能帮助我吗。

4

2 回答 2

2

减少内存使用可以做的一件事是消除中间StringBuilder和表示，并通过将返回的 from intostring直接从DataTableto写入：XmlDocumentXmlWriterXmlDocument.CreateNavigator.AppendChild()DataTable.WriteXml(XmlWriter)

var sd = new XmlDocument();
using (var writer = sd.CreateNavigator().AppendChild())
{
    dt.WriteXml(writer);
}

这也应该更高效，因为不再需要解析中间 XML 字符串。

（同样， aDataSet可以直接写入XmlDocumentusing DataSet.WriteXml(writer)。）

这是我用来在 x86 进程上验证这种方法的测试代码：

class TestClass
{
    internal static void Test()
    {
        // Verify the old and new algorithms generate identical XML
        foreach (var i in new[] { 0, 1, 2, 100 })
        {
            var old1 = TestStringWriteAndParse(i);
            var new1 = TestDirectWrite(i);

            if (old1.OuterXml != new1.OuterXml)
            {
                throw new InvalidOperationException("old1.OuterXml != new1.OuterXml");
            }
        }

        // Find a number of records that generate an out-of-memory exception whcn converting to an intermediate StringBuilder and string:
        uint size = 20000;
        try
        {
            while (size < int.MaxValue / 2)
            {
                TestStringWriteAndParse((int)size);
                size = checked(size * 2);
                GC.Collect();
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }

        GC.Collect();

        // Verify that number of records can be written directly using  XmlDocument.CreateNavigator.AppendChild()

        try
        {
            TestDirectWrite((int)size);
            Console.WriteLine("SUCCEEDED in writing {0} DataTable records to an XmlDocument", size);
        }
        catch (Exception ex)
        {
            Console.WriteLine("FAILED in writing {0} DataTable records to an XmlDocument:", size);
            Console.WriteLine(ex);
            throw;
        }
    }

    static DataTable GetData(int count)
    {
        var table = new DataTable();
        table.TableName = "Test";

        table.Columns.Add("Name", typeof(string));
        table.Columns.Add("Height", typeof(double));
        table.Columns.Add("NetWorth", typeof(decimal));

        for (int i = 0; i < count; i++)
        {
            DataRow row = table.NewRow();
            row["Name"] = "Bob Cratchit " + i.ToString();
            row["Height"] = 6.023;
            row["NetWorth"] = 101.01 + (10 * i);
            table.Rows.Add(row);
        }
        return table;
    }

    static XmlDocument TestDirectWrite(int count)
    {
        DataTable dt = GetData(count);// contains 20000 records.

        XmlDocument sd = new XmlDocument();
        using (XmlWriter writer = sd.CreateNavigator().AppendChild())
        {
            dt.WriteXml(writer);
        }

        return sd;
    }

    static XmlDocument TestStringWriteAndParse(int count)
    {
        DataTable dt = GetData(count);// contains 20000 records.

        StringWriter sw = new StringWriter();
        dt.WriteXml(sw);

        XmlDocument sd = new XmlDocument();
        sd.LoadXml(sw.ToString());

        return sd;
    }
}

在我的情况下，原始代码尝试序列化 1280000 条记录失败，而新代码成功。

于 2016-11-18T03:47:05.540 回答
0

您可以尝试使用文件流而不是字符串编写器。

FileStream stream = File.OpenRead(path);
XMLDocument doc = new XMLDocuemnt();
doc.Load(stream);
于 2016-11-17T16:28:47.103 回答