我在数据库中有大约 450K 记录,我必须在这些记录上进行模糊匹配。我已经实现了Parallel.foreach
循环来查看传入的请求是否与 450K 记录中的任何一个匹配。如果在搜索过程中找到任何匹配项,我会停止foreach
循环并以 true 将响应返回给调用者(它表示已找到匹配项)。
我正在调用数据库并将所有 450K 记录存储在静态对象中(在内存中)。for 的逻辑Parallel.foreach
运行良好。问题是确定它是否匹配需要将近 40-45 秒。
我计划parallel.foreach
在每个循环上运行多个 100k 数据,以减少总时间。parallel.foreach
我的挑战是,一旦我匹配到一个循环,我将如何停止执行其他循环parrallel.foreach
?
以下是我的单个代码parallel.foreach
:
public bool Checkmatch(CheckRequest request)
{
bool isPCCMatch = false;
//This will load all the data in Static object to avoid db call for each request
PolicyInformation.Initialize(request.IsCacheRefresher);
try
{
logger.Debug(MethodBase.GetCurrentMethod().Name + ": Matching alogorithm is started");
Parallel.ForEach(PolicyInformation.Policies, (policy, loopState) =>
{
double state=0, stateF=0;
if (!request.stateF)
{
state= GetPrecentageMatch(request.state, policy.state);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (state== OneFieldSSNPercentage)
{
isPCCMatch = true;
loopState.Stop();
}
}
else
{
stateF= GetPrecentageMatch(request.stateF, policy.stateF);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (stateF== OneFieldFEINPercentage)
{
isPCCMatch = true;
loopState.Stop();
}
}
double SEIN = GetPrecentageMatch(request.SEIN, policy.FEIN);
//this if condtion is for checkibnf first field condtion earily so that it will not get all the perecentage
if (SEIN == OneFieldSEINPercentage)
{
isPCCMatch = true;
loopState.Stop();
}
double WCIRB = GetPrecentageMatch(request.WCIRB, policy.WCIRB);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (WCIRB == OneFieldWCIRBPercentage)
{
isPCCMatch = true;
loopState.Stop();
}
double DUN = GetPrecentageMatch(request.DUNS, policy.DUNS);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (DUN == OneFieldDUNSPercentage)
{
isPCCMatch = true;
loopState.Stop();
}
double legalNames = GetPrecentageMatch(request.LegalName, policy.LegalName);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (legalNames == OneFieldLegalNamePercentage)
{
isPCCMatch = true;
loopState.Stop();
}
double tradeNames = GetPrecentageMatch(request.TradeName, policy.TradeName);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (tradeNames == OneFieldTradeNamePercentage)
{
isPCCMatch = true;
loopState.Stop();
}
double mailingname = GetPrecentageMatch(request.MailingName, policy.MailingName);
//this if condition is for checkibnf first field condition early so that it will not get all the percentage
if (mailingname == OneFieldMailingNamePercentage)
{
isPCCMatch = true;
loopState.Stop();
}
double ownerInfo = GetPrecentageMatch(request.OwnerName, policy.Ownership);
int partitalmatchcount = 0;
int addressmatchcount = 0;
// condtion 2
// get the partial count. if it is not more than 2 then compare the address an
if (GetPartialMatchFieldCount(SSN, FEIN, SEIN, DUN, legalNames, tradeNames, mailingname, ownerInfo, out partitalmatchcount) < 2)
{
// it will be hit if IsAddressmatch is true or count of partitalmatchcount & addressmatchcount is equal or more then 2
if (IsAddressmatch(request, policy, out addressmatchcount, partitalmatchcount) || (addressmatchcount + partitalmatchcount >= 2))
{
logger.Debug(MethodBase.GetCurrentMethod().Name + ": policy matched 2nd condition " + policy.ID);
isPCCMatch = true;
loopState.Stop();
}
}
else
{
isPCCMatch = true;
loopState.Stop();
}
// check for the 3 fields
if (GroupAmatchcount(SSN, FEIN, SEIN) + GroupBmatchcount(legalNames, tradeNames, mailingname) + GroupCMatchCount(request,
policy) + GroupDMatchCount(ownerInfo) >= 3)
{
logger.Debug(MethodBase.GetCurrentMethod().Name + ": policy matched 3rd condition " + policy.ID);
isPCCMatch = true;
loopState.Stop();
}
//else if ()
});
logger.Debug(MethodBase.GetCurrentMethod().Name + ": Matching algorithm is ended");
}
catch (Exception ex)
{
logger.ErrorException(MethodBase.GetCurrentMethod().Name, ex);
throw;
}
// }
return isPCCMatch;
}
我想为 100k 条记录运行上述循环,因此它将有 4 个parallel.foreach
循环。一旦我在一个循环上找到匹配项,我想停止其他parallel.foreach
循环的执行。