0

我一直在使用 LiteDb 来提高我的数据馈送的性能。提要在 Azure 数据库上运行，为了减少我们使用的 DTU 和连接数量（我在并行 forach 循环中执行一些逻辑），我从数据库中提取数据，将其放入 LiteDb 索引中，然后运行我的逻辑。我的大多数场景都很好。我首先做一个插入批量添加索引，然后以只读模式重新打开 litdb 并运行我的查询类似这样的东西

using (var db = new LiteDatabase("SiloLite.db"))
                {
                    db.DropCollection("SiloProduct");
                    var products = db.GetCollection<SiloProduct>("SiloProduct");
                    products.InsertBulk(_siloproductRepository.Table.ToList());
                    products.EnsureIndex(x => x.Sku);
                    db.Engine.EnsureIndex("SiloProduct", "UniqueProdId", "$.Sku+';'+$.ParentProductId", true);
                }
using (var db = new LiteDatabase($"Filename={AppDomain.CurrentDomain.BaseDirectory}\\SiloLite.db;mode=ReadOnly"))
                {
                    var products = db.GetCollection<SiloProduct>("SiloProduct");
                    var manufacturers = db.GetCollection<SiloManufacturer>("SiloManufacturer");
                    var productspecificationmap =
                        db.GetCollection<SiloProductSpecificationAttributeMapping>(
                            "SiloProductSpecificationAttributeMapping");
                    var specificationAttributes =
                        db.GetCollection<SiloSpecificationAttribute>("SiloSpecificationAttribute");
                    var specificationAttributeOptions =
                        db.GetCollection<SiloSpecificationAttributeOption>("SiloSpecificationAttributeOption");

                    Parallel.ForEach(katartlist, KatartItem =>
                    {
                        SaveData(KatartItem, specificationAttributes, specificationAttributeOptions, productspecificationmap,
                            manufacturers, products);
                    });
                    _logger.Information("Completed updating Product Specifications");
                }

它在所有场景中都非常有效。如果我尝试运行查询，然后尝试将该数据保存到 LiteDb 抱怨的另一个集合中。这就是我想要做的

var currentspecOptCollection = new List<SiloProductSpecificationAttributeMapping>(liteProdSpecCollection.Find(x => x.ProductId.Equals(productId)).ToList());
                    var specstoDelete = currentspecOptCollection
                                                          .Where(x => !specificationattributeoptionlist.Contains(x.SpecificationAttributeOptionId))
                                                          .ToList();
                    if (specstoDelete.Any())
                    {
                        foreach (var specattr in specstoDelete)
                        {
                            _specattrDeleteList.Add(specattr);
                        }

                    }

此逻辑在 Parallel.Foreach 内部执行，LiteDb 抛出此错误

System.InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
   at LiteDB.CacheService.GetPage(UInt32 pageID)
   at LiteDB.PageService.GetPage[T](UInt32 pageID)
   at LiteDB.QueryCursor.Fetch(TransactionService trans, DataService data, BsonReader bsonReader)
   at LiteDB.LiteEngine.Find(String collection, Query query, Int32 skip, Int32 limit)+MoveNext()
   at LiteDB.LiteEngine.Find(String collection, Query query, String[] includes, Int32 skip, Int32 limit)+MoveNext()
   at LiteDB.LiteCollection`1.Find(Query query, Int32 skip, Int32 limit)+MoveNext()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at CreateOrUpdateProductSpecifications(KatartItem katartItem, VarData vd, SiloManufacturer manufacturer, SiloProduct product, LiteCollection`1 liteSpecCollection, LiteCollection`1 liteSpecOptCollection, LiteCollection`1 liteProdSpecCollection) in *** 1399

不知道我在这里做错了什么。任何指针都会有所帮助

4

1 回答 1

1

我必须解决一个类似的问题，即我从 LiteDB 读取数据，然后将数据放入另一个列表中。我通过像这样编写并行执行来解决它：

public static async Task LoadDataFromDB()
{
    var tasks = new List<Task>();

    using (var db = new LiteDatabase(fullDbPath))
    {
        var bookChapterCollection = db.GetCollection<Chapter>("bookData");
        foreach (var volume in Volume.ListOfVolumes)
        {
            tasks.Add(Task.Factory.StartNew(() =>
            {
                var volumeChapterFromDB = bookChapterCollection.Find(x => x.chapterVolume == volume.volume).toList();
                return volumeChapterFromDB;
            }
            ,TaskCreationOptions.LongRunning)
           .ContinueWith(task => {
                BookChaptersBag.Add(task.Result);
                return task.Result;
            })
            );
        }
        await Task.WhenAll(tasks).ContinueWith(task =>
        {
            BookChaptersBag.CompleteAdding();
        });
    }
}

这里的 BookChaptersBag 是一个BlockingCollection<List<Chapter>>来自 System.Collections.Concurrent 命名空间的线程安全集合。在像上面那样编写我的并行执行之后，我停止了异常。

希望这可以帮助！

于 2019-08-05T16:56:27.107 回答