1

出于符合 GDPR 的原因，我正在尝试从语音转文本生成的杂乱数据中删除社会安全号码 (SSN)。这是一个示例字符串（翻译成英语，解释了为什么列出 SSN 时出现“和”）：

sample1 = "hello my name is sofie my social security number is thirteen zero four five and seventy eighteen seven and forty and I live on mountain street number twelve"

我的目标是删除该部分"thirteen ... forty "，同时保留字符串中可能出现的其他数字，从而导致：

sample1_wo_ssn = "hello my name is sofie my social security number is and I live on mountain street number twelve"

社会安全号码的长度可能会因数据的生成方式而异（3-10 个单独的数字）。

我的做法：

  1. 使用 dict 用数字替换书面数字
  2. 使用正则表达式查找只有空格或"and"分隔它们的 3 个或更多数字出现的位置，并将它们与这 3 个数字后面的任何数字一起删除。

这是我的代码：

import re

number_dict = {
    'zero': '0',
    'one': '1',
    'two': '2',
    'three': '3',
    'four': '4',
    'five': '5',
    'six': '6',
    'seven': '7',
    'eight': '8',
    'nine': '9',
    'ten': '10',
    'eleven': '11',
    'twelve': '12',
    'thirteen': '13',
    'fourteen': '14',
    'fifteen': '15',
    'sixteen': '16',
    'seventeen': '17',
    'eighteen': '18',
    'nineteen': '19',
    'twenty': '20',
    'thirty': '30',
    'forty': '40',
    'fifty': '50',
    'sixty': '60',
    'seventy': '70',
    'eighty': '80',
    'ninety': '90'
}


sample1 = "hello my name is sofie my social security number is thirteen zero four five and seventy eighteen seven and forty and I live on mountain street number twelve"
sample1_temp = [number_dict.get(item,item)  for item in sample1.split()]
sample1_numb = ' '.join(sample1_temp)
re_results = re.findall(r'(\d+ (and\s)?\d+ (and\s)?\d+\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?)', sample1_numb) 

print(re_results)

输出：

[('13 0 4 5 and 70 18 7 and 40 and ', '', '', '', '5', 'and ', '70', '', '18', '', '7', 'and ', '40', 'and ', '', '', '', '', '')]

这就是我卡住的地方。

在这个例子中，我可以做一些事情sample1_wh_ssn = re.sub(re_results[0][0],'',sample1_numb)来获得想要的结果，但这不会一概而论。

任何帮助将不胜感激。

4

1 回答 1

1

这是您当前逻辑的实现，即：

  • 将单词数字从1through99转换为数字
  • 删除以空格分隔的 3 个或更多数字的所有实例
  • 将数字两位数的数字转换回单词。

学分：

请参阅Python 代码：

import re

number_words = [ "zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]
number_words_tens =[ "twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety" ]
number_words_rx = re.compile(r'\b(?:(?:{0})?(?:{1})|(?:{0}))\b'.format("|".join(number_words_tens),"|".join(number_words)))
main_rx = re.compile(r'\s*\d+(?:\s+(?:and\s+)?\d+){2,}')
numbers_1_99 = number_words
numbers_1_99.extend(tens if ones == "zero" else (tens + "-" + ones) # stackoverflow.com/a/8982279/3832970
    for tens in "twenty thirty forty fifty sixty seventy eighty ninety".split()
    for ones in numbers_1_99[0:10])

def text2int(textnum, numwords={}): # stackoverflow.com/a/493788/3832970
    units = [
        "zero", "one", "two", "three", "four", "five", "six", "seven", "eight",
        "nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen",
        "sixteen", "seventeen", "eighteen", "nineteen",
    ]
    tens = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety"]
    numwords["and"] = (1, 0)
    for idx, word in enumerate(units):
        numwords[word] = (1, idx)
    for idx, word in enumerate(tens):
        numwords[word] = (1, idx * 10)
    current = result = 0
    for word in textnum.split():
        if word not in numwords:
          raise Exception("Illegal word: " + word)

        scale, increment = numwords[word]
        current = current + increment

    return result + current
sample1 = "hello my name is sofie my social security number is thirteen zero four five and seventy eighteen seven and forty and I live on mountain street number twelve"
sample1 = number_words_rx.sub(lambda x: str(text2int(x.group())), sample1)
re_results = main_rx.sub('', sample1)
print( re.sub(r'\d{1,2}', lambda x: numbers_1_99[int(x.group())], re_results) )

输出：hello my name is sofie my social security number is and I live on mountain street number twelve

于 2020-04-02T13:38:00.397 回答