0

我想将 1000000 个文档插入 RavenDB。

class Program
{
        private static string serverName;
        private static string databaseName;

        private static DocumentStore documentstore;
        private static IDocumentSession _session;

        static void Main(string[] args)
        {

            Console.WriteLine("Start...");

            serverName = ConfigurationManager.AppSettings["ServerName"];
            databaseName = ConfigurationManager.AppSettings["Database"];

            documentstore = new DocumentStore { Url = serverName };
            documentstore.Initialize();

            Console.WriteLine("Initial Databse...");

            _session = documentstore.OpenSession(databaseName);

            for (int i = 0; i < 1000000; i++)
            {
                var person = new Person()    
                {
                    Fname = "Meysam" + i,
                    Lname = " Savameri" + i,
                    Bdate = DateTime.Now,
                    Salary = 6001 + i,
                    Address = "BITS provides one foreground and three background priority levels that" +
                              "you can use to prioritize transBfer jobs. Higher priority jobs preempt"+
                              "lower priority jobs. Jobs at the same priority level share transfer time,"+
                              "which prevents a large job from blocking small jobs in the transfer"+
                              "queue. Lower priority jobs do not receive transfer time until all the "+
                              "higher priority jobs are complete or in an error state. Background"+
                              "transfers are optimal because BITS uses idle network bandwidth to"+
                              "transfer the files. BITS increases or decreases the rate at which files "+
                              "are transferred based on the amount of idle network bandwidth that is"+
                              "available. If a network application begins to consume more bandwidth,"+
                              "BITS decreases its transfer rate to preserve the user's interactive"+
                              "experience. BITS supports multiple foreground jobs and one background"+
                              "transfer job at the same time.",
                    Email = "Meysam" + i + "@hotmail.com",
                };

                _session.Store(person);

                Console.ForegroundColor = ConsoleColor.Green;
                Console.WriteLine("Count:" + i);
                Console.ForegroundColor = ConsoleColor.White;
            }

            Console.WriteLine("Commit...");

            _session.SaveChanges();
            documentstore.Dispose();

            _session.Dispose();

            Console.WriteLine("Complete...");
            Console.ReadLine();
        }
    }

但会话不保存更改，我收到一个错误：

mscorlib.dll 中出现“System.OutOfMemoryException”类型的未处理异常

4

2 回答 2

8

文档会话旨在处理少量请求。相反，尝试分批插入 1024。之后，处理会话并创建一个新会话。你得到一个的原因OutOfMemoryException是因为文档会话缓存了所有组成对象以提供一个工作单元，这就是为什么你应该在插入一个批次后处理会话。

一个巧妙的方法是使用Batch linq 扩展：

foreach (var batch in Enumerable.Range(1, 1000000)
 .Select(i => new Person { /* set properties */ })
 .Batch(1024))
{
 using (var session = documentstore.OpenSession())
 {
   foreach (var person in batch)
   {
     session.Store(person);
   }
   session.SaveChanges();
 }
}

Enumerable.Range和的实现Batch都是惰性的，不会将所有对象都保存在内存中。

于 2012-10-13T17:48:00.487 回答
1

RavenDB 还有一个批量 API，无需额外的 LINQ 扩展即可执行类似的操作：

using (var bulkInsert = store.BulkInsert())
{
    for (int i = 0; i < 1000 * 1000; i++)
    {
        bulkInsert.Store(new User
            {
                Name = "Users #" + i
            });
    }
}

Note.SaveChanges()不会被调用，并且会在达到批量大小（在BulkInsert()需要时定义）或被bulkInsert处理时调用。

于 2015-01-21T22:26:00.750 回答