-3

目前我正在使用以下代码......它仅适用于 300 行文本文件......执行此程序代码需要 2 分钟......但我的文本文件有超过 200k 行（行） ，所以这段代码不适用于该文件......所以请任何人帮助我解决这个问题......提前谢谢......

string[] source = System.IO.File.ReadAllLines(@"C:\Documents and Settings\finaloutput.txt");      

var q1 = (from line in source
          let fields = line.Split(',')
          select new
          {
              autoid = fields[0],
              ATMID = fields[4],
              DATE = fields[2],
              TIME = fields[3],
              CARDNo = fields[5],
              TRANSId = fields[6],
              SEQNo = fields[7],
              TRANSIT = fields[8],
              CheckNo = fields[9],
              CATEGORY = fields[10],
              SCORE = fields[11],
              //THRESHOLD = fields[12]
          });


    var ids = (from d in q1
               where d.CATEGORY != "Accepted"
               group d by new { d.ATMID, d.DATE, d.CARDNo, d.TRANSIT, d.CheckNo } into grp
               select grp.Min(x => x.autoid));


    var toDelete = (from d in q1
                    where !ids.Contains(d.autoid) && d.CATEGORY != "Accepted"
                    select d.autoid);

    // source1.DeleteOnSubmit(toDelete);

    var distinct = (from d in q1
                    where !toDelete.Contains(d.autoid)
                    select d);



    // Makes a list of the DeletedFields  
    // var list_Of_CSV_ItemsDeleted = distinct.Select(x => string.Join(",", x.autoid));

    // Makes a list of the distinct Fields  
    var list_Of_CSV_ItemsDistinct = distinct.Select(x => string.Join(",", x.autoid, x.ATMID, x.DATE, x.TIME, x.CARDNo, x.TRANSId, x.SEQNo, x.TRANSIT, x.CheckNo, x.CATEGORY, x.SCORE)); 
    System.IO.File.WriteAllLines(@"C:\Documents and Settings\distict1.txt", list_Of_CSV_ItemsDistinct);
4

1 回答 1

1

我不会为你重写这个，但你需要做的一件事是利用延迟执行。考虑这段代码：

var enumerable = File.ReadLines(filePath);

这会返回一个IEnumerable<string>，因此它只会在您请求时从文件中读取一行。现在考虑这段代码：

var next100 = enumerable.Take(100);

那将需要 100 行代码，然后让您使用它们。这就是你必须这样做的方式。您仍然可以使用几乎相同的 LINQ 查询，但一次只能使用一个部分。

所以，而不是这样的事情：

var q1 = (from line in source ...

它可能必须是这样的：

var q1 = (from line in source.Take(100) ...
于 2013-06-14T15:11:57.390 回答