我正在编写一个 Python 函数来将文本拆分为单词,而忽略指定的标点符号。这是一些工作代码。我不相信从列表中构造字符串(代码中的 buf = [] )是有效的。有没有人建议更好的方法来做到这一点?
def getwords(text, splitchars=' \t|!?.;:"'):
"""
Generator to get words in text by splitting text along specified splitchars
and stripping out the splitchars::
>>> list(getwords('this is some text.'))
['this', 'is', 'some', 'text']
>>> list(getwords('and/or'))
['and', 'or']
>>> list(getwords('one||two'))
['one', 'two']
>>> list(getwords(u'hola unicode!'))
[u'hola', u'unicode']
"""
splitchars = set(splitchars)
buf = []
for char in text:
if char not in splitchars:
buf.append(char)
else:
if buf:
yield ''.join(buf)
buf = []
# All done. Yield last word.
if buf:
yield ''.join(buf)