Currently one my DAL methods uses the ExecuteXmlReader command to read and append to a StringBuilder object and return XML as a string. I realize this isn't the best way to handle very large files (200MB+) as when the ToString() method is called on StringBuilder it throws an out of memory exception.
There are certain constraints I need to adhere by - I cannot return a XMLReader out of the DAL (db connection stays open, etc), this XML will have to be written to file and keeping it's formatting for various reasons, currently using an XMLTextWriter's WriteRaw method for that purpose.
What would be the best data type to return from the DAL to make all this work with large XML files? I explored the options of using a DataSet, Memory Stream and a Byte Array - but each have their shortfalls. I am looking for the best practice and some code samples would be helpful.