0

在匹配下面给出的数据中的短语时需要帮助，我需要匹配来自 TextA 和 TextB 的短语。

以下代码没有帮助我做到这一点我该如何解决这个我有100个要匹配

#排序混乱的短语

def sorts(string_value):
    sorted_string = sorted(string_value.split())
    sorted_string = ' '.join(sorted_string)
    return sorted_string

#删除字符串中的标点符号

punc = '''!()-[]{};:'"\,<>./?@#$%^&*_~'''

def punt(test_str):
    for ele in test_str:
        if ele in punc:
            test_str = test_str.replace(ele, "")
    return(test_str)

#匹配字符串

def lets_match(x):

    for text1 in TextA:
        for text2 in TextB:
            try:
                if sorts(punt(x[text1.casefold()])) == sorts(punt(x[text2.casefold()])):
                    return True
            except:
                continue
    return False
df['result'] = df.apply(lets_match,axis =1)

即使在实现字符串排序、删除标点符号和区分大小写之后，我仍然认为这些字符串不匹配。我在这里遗漏了一些东西可以帮助我实现它

4

3 回答 3

2

实际上，您可以使用difflib匹配两个文本，这是您可以尝试的：

from difflib import SequenceMatcher

def similar(a, b):
    a=str(a).lower()
    b=str(b).lower()
    return SequenceMatcher(None, a, b).ratio()

def lets_match(d):
    print(d[0]," --- ",d[1])
    result=similar(d[0],d[1])
    print(result)
    if result>0.6:
        return True
    else:
        return False
    
df["result"]=df.apply(lets_match,axis =1)

你可以发挥if result>0.6价值。

有关difflib您的更多信息，请访问此处。还有其他序列匹配器也像textdistance但我发现它很容易所以我尝试了这个。

于 2021-08-02T09:34:52.577 回答
1

我认为如果没有字符串距离概念，您将无法做到这一点，您可以做的是使用，例如记录链接。

我不会详细介绍，但我会向您展示这个案例的使用示例。

import pandas as pd 
import recordlinkage as rl 
from recordlinkage.preprocessing import clean

# creating first dataframe
df_text_a = pd.DataFrame({
    "Text A":[
        "AKIL KUMAR SINGH",
        "OUSMANI DJIBO",
        "PETER HRYB",
        "CNOC LIMITED",
        "POLY NOVA INDUSTRIES LTD",
        "SAM GAWED JR",
        "ADAN GENERAL LLC",
        "CHINA MOBLE LIMITED",
        "CASTAR CO., LTD.",
        "MURAN",
        "OLD SAROOP FOR CAR SEAT COVERS",
        "CNP HEALTHCARE, LLC",
        "GLORY PACK LTD",
        "AUNCO VENTURES",
        "INTERNATIONAL COMPANY",
        "SAMEERA HEAT AND ENERGY FUND"]
                }
            )

# creating second dataframe
df_text_b = pd.DataFrame({
        "Text B":[
            "Singh, Akil Kumar",
            "DJIBO, Ousmani Illiassou",
            "HRYB, Peter",
            "CNOOC LIMITED",
            "POLYNOVA INDUSTRIES LTD. ",
            "GAWED, SAM",
            "ADAN GENERAL TRADING FZE",
            "CHINA MOBILE LIMITED",
            "CASTAR GROUP CO., LTD.",
            "MURMAN    ",
            "Old Saroop for Car Seat Covers",
            "CNP HEATHCARE, LLC",
            "GLORY PACK LTD.",
            "AUNCO VENTURE",
            "INTL COMPANY",
            "SAMEERA HEAT AND ENERGY PROPERTY FUND"
            ]
        }
)

# preprocessing in very important on results, you have to find which fit well on yuor problem.
cleaned_a = pd.DataFrame(clean(df_text_a["Text A"], lowercase=True))
cleaned_b = pd.DataFrame(clean(df_text_b["Text B"], lowercase=True))

# creating an indexing which will be used for comprison, you have various type of indexing, watch documentation.

indexer = rl.Index()
indexer.full()
# generating all passible pairs
pairs = indexer.index(cleaned_a, cleaned_b)

# starting evaluation phase
compare = rl.Compare(n_jobs=-1)  
compare.string("Text A", "Text B", method='jarowinkler', label = 'text')
matches = compare.compute(pairs, cleaned_a, cleaned_b)

matchs 现在是一个 MultiIndex DataFrame，您接下来要做的是通过第一个索引查找第二个索引上的所有最大值。所以你会得到你需要的结果。

在距离、索引和/或预处理方面可以改进结果。

于 2021-08-02T09:35:13.603 回答
1

使用模糊匹配库有什么问题吗？鉴于上述数据相对相似，该实现非常简单并且运行良好。我在没有预处理的情况下执行了以下操作。

    import pandas as pd
    """ Install the libs below via terminal:

            $pip install fuzzywuzzy
            $pip install python-Levenshtein
    """

    from fuzzywuzzy import fuzz
    from fuzzywuzzy import process


    #creating the data frames
        text_a = ['AKIL KUMAR SINGH','OUSMANI DJIBO','PETER HRYB','CNOC LIMITED','POLY NOVA INDUSTRIES LTD','SAM GAWED JR','ADAN GENERAL LLC','CHINA MOBLE LIMITED','CASTAR CO., LTD.','MURAN','OLD SAROOP FOR CAR SEAT COVERS','CNP HEALTHCARE, LLC','GLORY PACK LTD','AUNCO VENTURES','INTERNATIONAL COMPANY','SAMEERA HEAT AND ENERGY FUND']
        text_b = ['Singh, Akil Kumar','DJIBO, Ousmani Illiassou','HRYB, Peter','CNOOC LIMITED','POLYNOVA INDUSTRIES LTD.','GAWED, SAM','ADAN GENERAL TRADING FZE','CHINA MOBILE LIMITED','CASTAR GROUP CO., LTD.','MURMAN','Old Saroop for Car Seat Covers','CNP HEATHCARE, LLC','GLORY PACK LTD.','AUNCO VENTURE','INTL COMPANY','SAMEERA HEAT AND ENERGY PROPERTY FUND']
        df_text_a = pd.DataFrame(text_a, columns=['text_a'])
        df_text_b = pd.DataFrame(text_b, columns=['text_b'])
        
        def lets_match(txt: str, chklist: list) -> str: 
            return process.extractOne(txt, chklist, scorer=fuzz.token_set_ratio)


    #match Text_A against Text_B
        result_txt_ab = df_text_a.apply(lambda x: lets_match(str(x), text_b), axis=1, result_type='expand')
        result_txt_ab.rename(columns={0:'Return Match', 1:'Match Value'}, inplace=True)
        df_text_a[result_txt_ab.columns]=result_txt_ab
        df_text_a

    text_a  Return Match    Match Value
    0   AKIL KUMAR SINGH    Singh, Akil Kumar   100
    1   OUSMANI DJIBO   DJIBO, Ousmani Illiassou    72
    2   PETER HRYB  HRYB, Peter 100
    3   CNOC LIMITED    CNOOC LIMITED   70
    4   POLY NOVA INDUSTRIES LTD    POLYNOVA INDUSTRIES LTD.    76
    5   SAM GAWED JR    GAWED, SAM  100
    6   ADAN GENERAL LLC    ADAN GENERAL TRADING FZE    67
    7   CHINA MOBLE LIMITED CHINA MOBILE LIMITED    79
    8   CASTAR CO., LTD.    CASTAR GROUP CO., LTD.  81
    9   MURAN   SAMEERA HEAT AND ENERGY PROPERTY FUND   41
    10  OLD SAROOP FOR CAR SEAT COVERS  Old Saroop for Car Seat Covers  100
    11  CNP HEALTHCARE, LLC CNP HEATHCARE, LLC  58
    12  GLORY PACK LTD  GLORY PACK LTD. 100
    13  AUNCO VENTURES  AUNCO VENTURE   56
    14  INTERNATIONAL COMPANY   INTL COMPANY    74
    15  SAMEERA HEAT AND ENERGY FUND    SAMEERA HEAT AND ENERGY PROPERTY FUND   86

    #match Text_B against Text_A
    result_txt_ba= df_text_b.apply(lambda x: lets_match(str(x), text_a), axis=1, result_type='expand')
    result_txt_ba.rename(columns={0:'Return Match', 1:'Match Value'}, inplace=True)
    df_text_b[result_txt_ba.columns]=result_txt_ba
    df_text_b

text_b  Return Match    Match Value
0   Singh, Akil Kumar   AKIL KUMAR SINGH    100
1   DJIBO, Ousmani Illiassou    OUSMANI DJIBO   100
2   HRYB, Peter PETER HRYB  100
3   CNOOC LIMITED   CNOC LIMITED    74
4   POLYNOVA INDUSTRIES LTD.    POLY NOVA INDUSTRIES LTD    74
5   GAWED, SAM  SAM GAWED JR    86
6   ADAN GENERAL TRADING FZE    ADAN GENERAL LLC    86
7   CHINA MOBILE LIMITED    CHINA MOBLE LIMITED 81
8   CASTAR GROUP CO., LTD.  CASTAR CO., LTD.    100
9   MURMAN  ADAN GENERAL LLC    33
10  Old Saroop for Car Seat Covers  OLD SAROOP FOR CAR SEAT COVERS  100
11  CNP HEATHCARE, LLC  CNP HEALTHCARE, LLC 56
12  GLORY PACK LTD. GLORY PACK LTD  100
13  AUNCO VENTURE   AUNCO VENTURES  53
14  INTL COMPANY    INTERNATIONAL COMPANY   50
15  SAMEERA HEAT AND ENERGY PROPERTY FUND   SAMEERA HEAT AND ENERGY FUND    100
于 2021-08-02T17:18:32.677 回答