英语有几个缩略词。例如：
you've -> you have
he's -> he is
当您进行自然语言处理时，这些有时会引起头痛。是否有 Python 库可以扩展这些收缩？
问问题
47722 次
8 回答
62
我把维基百科的收缩到扩展页面变成了一个 python 字典（见下文）
请注意，正如您所料，您在查询字典时肯定要使用双引号：
此外，我在维基百科页面中留下了多个选项。随意修改它。请注意，正确展开的消歧将是一个棘手的问题！
contractions = {
"ain't": "am not / are not / is not / has not / have not",
"aren't": "are not / am not",
"can't": "cannot",
"can't've": "cannot have",
"'cause": "because",
"could've": "could have",
"couldn't": "could not",
"couldn't've": "could not have",
"didn't": "did not",
"doesn't": "does not",
"don't": "do not",
"hadn't": "had not",
"hadn't've": "had not have",
"hasn't": "has not",
"haven't": "have not",
"he'd": "he had / he would",
"he'd've": "he would have",
"he'll": "he shall / he will",
"he'll've": "he shall have / he will have",
"he's": "he has / he is",
"how'd": "how did",
"how'd'y": "how do you",
"how'll": "how will",
"how's": "how has / how is / how does",
"I'd": "I had / I would",
"I'd've": "I would have",
"I'll": "I shall / I will",
"I'll've": "I shall have / I will have",
"I'm": "I am",
"I've": "I have",
"isn't": "is not",
"it'd": "it had / it would",
"it'd've": "it would have",
"it'll": "it shall / it will",
"it'll've": "it shall have / it will have",
"it's": "it has / it is",
"let's": "let us",
"ma'am": "madam",
"mayn't": "may not",
"might've": "might have",
"mightn't": "might not",
"mightn't've": "might not have",
"must've": "must have",
"mustn't": "must not",
"mustn't've": "must not have",
"needn't": "need not",
"needn't've": "need not have",
"o'clock": "of the clock",
"oughtn't": "ought not",
"oughtn't've": "ought not have",
"shan't": "shall not",
"sha'n't": "shall not",
"shan't've": "shall not have",
"she'd": "she had / she would",
"she'd've": "she would have",
"she'll": "she shall / she will",
"she'll've": "she shall have / she will have",
"she's": "she has / she is",
"should've": "should have",
"shouldn't": "should not",
"shouldn't've": "should not have",
"so've": "so have",
"so's": "so as / so is",
"that'd": "that would / that had",
"that'd've": "that would have",
"that's": "that has / that is",
"there'd": "there had / there would",
"there'd've": "there would have",
"there's": "there has / there is",
"they'd": "they had / they would",
"they'd've": "they would have",
"they'll": "they shall / they will",
"they'll've": "they shall have / they will have",
"they're": "they are",
"they've": "they have",
"to've": "to have",
"wasn't": "was not",
"we'd": "we had / we would",
"we'd've": "we would have",
"we'll": "we will",
"we'll've": "we will have",
"we're": "we are",
"we've": "we have",
"weren't": "were not",
"what'll": "what shall / what will",
"what'll've": "what shall have / what will have",
"what're": "what are",
"what's": "what has / what is",
"what've": "what have",
"when's": "when has / when is",
"when've": "when have",
"where'd": "where did",
"where's": "where has / where is",
"where've": "where have",
"who'll": "who shall / who will",
"who'll've": "who shall have / who will have",
"who's": "who has / who is",
"who've": "who have",
"why's": "why has / why is",
"why've": "why have",
"will've": "will have",
"won't": "will not",
"won't've": "will not have",
"would've": "would have",
"wouldn't": "would not",
"wouldn't've": "would not have",
"y'all": "you all",
"y'all'd": "you all would",
"y'all'd've": "you all would have",
"y'all're": "you all are",
"y'all've": "you all have",
"you'd": "you had / you would",
"you'd've": "you would have",
"you'll": "you shall / you will",
"you'll've": "you shall have / you will have",
"you're": "you are",
"you've": "you have"
}
于 2013-11-05T17:20:48.657 回答
26
上面的答案将非常有效，并且对于模棱两可的收缩可能更好（尽管我认为没有那么多模棱两可的情况）。我会使用一些更易读和更容易维护的东西：
import re
def decontracted(phrase):
# specific
phrase = re.sub(r"won\'t", "will not", phrase)
phrase = re.sub(r"can\'t", "can not", phrase)
# general
phrase = re.sub(r"n\'t", " not", phrase)
phrase = re.sub(r"\'re", " are", phrase)
phrase = re.sub(r"\'s", " is", phrase)
phrase = re.sub(r"\'d", " would", phrase)
phrase = re.sub(r"\'ll", " will", phrase)
phrase = re.sub(r"\'t", " not", phrase)
phrase = re.sub(r"\'ve", " have", phrase)
phrase = re.sub(r"\'m", " am", phrase)
return phrase
test = "Hey I'm Yann, how're you and how's it going ? That's interesting: I'd love to hear more about it."
print(decontracted(test))
# Hey I am Yann, how are you and how is it going ? That is interesting: I would love to hear more about it.
它可能有一些我没有想到的缺陷。
从我的另一个答案转贴
于 2017-11-03T08:38:37.473 回答
18
您不需要库，例如可以使用 reg exp。
>>> import re
>>> contractions_dict = {
... 'didn\'t': 'did not',
... 'don\'t': 'do not',
... }
>>> contractions_re = re.compile('(%s)' % '|'.join(contractions_dict.keys()))
>>> def expand_contractions(s, contractions_dict=contractions_dict):
... def replace(match):
... return contractions_dict[match.group(0)]
... return contractions_re.sub(replace, s)
...
>>> expand_contractions('You don\'t need a library')
'You do not need a library'
于 2013-11-05T13:39:52.550 回答
18
我为此找到了一个库，contractions它非常简单。
import contractions
print(contractions.fix("you've"))
print(contractions.fix("he's"))
输出：
you have
he is
于 2019-02-13T07:21:43.573 回答
11
这是一个非常酷且易于使用的库，用于 https://pypi.python.org/pypi/pycontractions/1.0.1。
使用示例（详见链接）：
from pycontractions import Contractions
# Load your favorite word2vec model
cont = Contractions('GoogleNews-vectors-negative300.bin')
# optional, prevents loading on first expand_texts call
cont.load_models()
out = list(cont.expand_texts(["I'd like to know how I'd done that!",
"We're going to the zoo and I don't think I'll be home for dinner.",
"Theyre going to the zoo and she'll be home for dinner."], precise=True))
print(out)
您还需要 GoogleNews-vectors-negative300.bin，在上面的 pycontractions 链接中下载链接。*python3 中的示例代码。
于 2018-04-06T07:59:23.350 回答
4
我想在这里对 alko 的回答做一点补充。如果您查看维基百科，所提到的英语语言缩写的数量少于 100。当然，在实际情况下，这个数字可能会更多。但是，我很确定 200 到 300 个单词就可以用于英语收缩词。现在，您是否想为那些获得一个单独的库（不过，我认为您正在寻找的东西实际上并不存在）？但是，您可以使用字典和正则表达式轻松解决此问题。我建议使用一个不错的标记器作为自然语言工具包，其余的你自己实现应该没有问题。
于 2013-11-05T13:47:49.427 回答
1
def expand_contractions(text, contraction_mapping=CONTRACTION_MAP):
# contraction_mapping is a dictionary of words having the compact form
contractions_pattern = re.compile('({})'.format('|'.join(contraction_mapping.keys())),flags=re.IGNORECASE|re.DOTALL)
def expand_match(contraction):
match = contraction.group(0)
first_char = match[0]
expanded_contraction = contraction_mapping.get(match) \
if contraction_mapping.get(match) \
else contraction_mapping.get(match.lower())
expanded_contraction = first_char+expanded_contraction[1:]
return expanded_contraction
expanded_text = contractions_pattern.sub(expand_match, text)
expanded_text = re.sub("'", "", expanded_text)
return expanded_text
于 2020-06-23T18:08:43.117 回答
0
尽管这是一个老问题，但我想我还是回答一下，因为据我所知，仍然没有真正的解决方案。
我不得不在一个相关的 NLP 项目上解决这个问题，我决定解决这个问题，因为这里似乎没有任何东西。如果您有兴趣，可以查看我的扩展器 github 存储库。
这是一个基于 NLTK、Stanford Core NLP 模型（您必须单独下载）和上一个答案中的字典的相当糟糕的优化（我认为）程序。所有必要的信息都应该在自述文件和大量注释的代码中。我知道带注释的代码是死代码，但这正是我为自己保持清晰而编写的方式。
中的示例输入expander.py是以下句子：
["I won't let you get away with that", # won't -> will not
"I'm a bad person", # 'm -> am
"It's his cat anyway", # 's -> is
"It's not what you think", # 's -> is
"It's a man's world", # 's -> is and 's possessive
"Catherine's been thinking about it", # 's -> has
"It'll be done", # 'll -> will
"Who'd've thought!", # 'd -> would, 've -> have
"She said she'd go.", # she'd -> she would
"She said she'd gone.", # she'd -> had
"Y'all'd've a great time, wouldn't it be so cold!", # Y'all'd've -> You all would have, wouldn't -> would not
" My name is Jack.", # No replacements.
"'Tis questionable whether Ma'am should be going.", # 'Tis -> it is, Ma'am -> madam
"As history tells, 'twas the night before Christmas.", # 'Twas -> It was
"Martha, Peter and Christine've been indulging in a menage-à-trois."] # 've -> have
输出到哪个
["I will not let you get away with that",
"I am a bad person",
"It is his cat anyway",
"It is not what you think",
"It is a man's world",
"Catherine has been thinking about it",
"It will be done",
"Who would have thought!",
"She said she would go.",
"She said she had gone.",
"You all would have a great time, would not it be so cold!",
"My name is Jack.",
"It is questionable whether Madam should be going.",
"As history tells, it was the night before Christmas.",
"Martha, Peter and Christine have been indulging in a menage-à-trois."]
因此，对于这一小组测试语句，我想出了一些边缘情况来测试，它运行良好。
由于这个项目现在已经失去了重要性，我不再积极开发这个了。对此项目的任何帮助将不胜感激。要做的事情都写在 TODO 列表中。或者，如果您对如何改进我的 python 有任何提示，我也会非常感激。
于 2017-10-18T10:23:02.170 回答