c# - C# Linq 大连接内存不足

Question

我有以下加入声明：

  var query = from holding in dHoldList
              join client in clientList on
                new { holding.ClientNo }
                equals new { client.ClientNo } into clj
              from cl in clj.DefaultIfEmpty()
              join matter in matterList on 
                new { holding.ClientNo } 
                equals new { matter.ClientNo } into mlj
              from ml in mlj.DefaultIfEmpty()
              join stk in stockList on
                new { holding.Sedol }
                equals new { stk.Sedol } into slj
              from sl in slj.DefaultIfEmpty()
              select
                new GeneralHoldingsReport()
                {
                  ClientNo = holding.ClientNo,
                  Depot = holding.Depot,
                  HoldingSedol = holding.Sedol,
                  Value = holding.ValueOfStock,
                  NoOfUnits = holding.QuantityHeld,
                  ClientName = (cl == null ? null : cl.ClientName),
                  CountryOfResidence = (cl == null ? null : cl.CountryOfResidence),
                  BG = (cl == null ? null : cl.BusinessGetter),
                  ClientStockValue = (ml == null ? 0 : ml.FullValueOfPortfolio),
                  StockName = (sl == null ? null : sl.R1.Trim() + " " + sl.R2.Trim())
                };


  var reportList = query.ToList();

但是，当它运行时，我收到内存不足异常错误。

我要求 dHoldList 是主表，所有其他表都加入其中（即，如果与 dHoldList 中的每条记录相关的其他表中的数据匹配，则返回数据，如果不只是空白的话。）我相信我正在这样做正确，但显然不是。

这些列表中的每一个都包含大约 300k 行，除了客户端只有 30k，因此这可能会导致这里的一些问题。

score 2 · Accepted Answer

您在联接中实例化了许多无用的匿名类型。您可以按如下方式编写它们：
```
join client in clientList on holding.ClientNo equals client.ClientNo into clj 
```
也许这已经解决了你的记忆问题？
这仅适用于 LINQ to Objects。如果您的查询像 EF 或 LINQ to SQL 一样被转换为 SQL，则不会创建这些匿名类型。

考虑重组查询以使其更具可读性：

var query = 
    from holding in dHoldList 
    join client in clientList on holding.ClientNo equals client.ClientNo into clj 
    join matter in matterList on holding.ClientNo equals matter.ClientNo into mlj 
    join stk in stockList on holding.Sedol equals stk.Sedol into slj 

    from cl in clj.DefaultIfEmpty() 
    from ml in mlj.DefaultIfEmpty() 
    from sl in slj.DefaultIfEmpty() 
    select new GeneralHoldingsReport() 
    { 
        ClientNo = holding.ClientNo, 
        Depot = holding.Depot, 
        HoldingSedol = holding.Sedol, 
        Value = holding.ValueOfStock, 
        NoOfUnits = holding.QuantityHeld, 
        ClientName = (cl == null ? null : cl.ClientName), 
        CountryOfResidence = (cl == null ? null : cl.CountryOfResidence), 
        BG = (cl == null ? null : cl.BusinessGetter), 
        ClientStockValue = (ml == null ? 0 : ml.FullValueOfPortfolio), 
        StockName = (sl == null ? null : sl.R1.Trim() + " " + sl.R2.Trim()) 
    };

这不会对内存占用产生任何影响，但会影响可读性和可维护性。

如果我要编写代码来实现您的目标，尤其是涉及这么多对象，我不会使用join. 我会使用哈希表。这将大大加快。

像这样的东西（未经测试）：

var clients = clientList.ToDictionary(x => x.ClientNo);
var matters = matterList.ToDictionary(x => x.ClientNo);
var stocks = stockList.ToDictionary(x => x.Sedol);

var reportList = new List<GeneralHoldingsReport>(dHoldList.Count);
Client client;
Matter matter;
Stock stock;    

foreach(var holding in dHoldList)
{
    if(!clients.TryGetValue(holding.ClientNo, out client))
        client = null;
    if(!matters.TryGetValue(holding.ClientNo, out matter))
        matter = null;
    if(!stocks.TryGetValue(holding.Sedol, out stock))
        stock = null;
    reportList.Add(new GeneralHoldingsReport()     
                   {     
                       ClientNo = holding.ClientNo,     
                       Depot = holding.Depot,     
                       HoldingSedol = holding.Sedol,     
                       Value = holding.ValueOfStock,     
                       NoOfUnits = holding.QuantityHeld,     
                       ClientName = (client == null ? null :
                                     client.ClientName),     
                       CountryOfResidence = (client == null ? null : 
                                             client.CountryOfResidence),     
                       BG = (client == null ? null : 
                             client.BusinessGetter),     
                       ClientStockValue = (matter == null ? 0 : 
                                           matter.FullValueOfPortfolio),     
                       StockName = (stock == null ? null : 
                                    stock.R1.Trim() + " " 
                                    + stock.R2.Trim())     
                   });
}

ToDictionary如果每个客户编号有超过一个客户或事项，并且每个 sedol 超过一个股票，则使用将导致崩溃。

c# - C# Linq 大连接内存不足

1 回答 1

Related

Reference