0

NLTK 附带一些语料库样本：http: //nltk.googlecode.com/svn/trunk/nltk_data/index.xml

我只想有没有编码的文本。我不知道如何提取这些内容。我要提取的是

1) nps_chat：解压后文件名类似于 10-19-20s_706posts.xml。此类文件是 XML 格式，例如：

<Posts>
<Post class="Statement" user="10-19-20sUser7">now im left with this gay name<terminals>

                <t pos="RB" word="now"/>
                <t pos="PRP" word="im"/>
                <t pos="VBD" word="left"/>
                <t pos="IN" word="with"/>
                <t pos="DT" word="this"/>
                <t pos="JJ" word="gay"/>
                <t pos="NN" word="name"/>
            </terminals>

        </Post>
            ...
            ...

我只想要那个实际的帖子：

now im left with this gay name

在本地磁盘中剥离编码后，如何在 NLTK 或（无论如何）保存裸帖子？

2) 总机成绩单。此类文件（文件名是解压后的discourse）包含以下格式。我想要的是去除前面的标记：

o A.1 utt1: Okay, /
qy A.1 utt2: have you ever served as a juror? /
ng B.2 utt1: Never. /
sd^e B.2 utt2: I've never been served on the jury, never been called up in a jury, although some of my friends have been jurors. /
b A.3 utt1: Uh-huh. /
sd A.3 utt2: I never have either. /
% B.4 utt1: You haven't, {F huh. } /
...
...

我只想拥有：

Okay, /
have you ever served as a juror? /
Never. /
I've never been served on the jury, never been called up in a jury, although some of my friends have been jurors. /
Uh-huh. /
I never have either. /
You haven't, {F huh. } /
...
...

非常感谢您提前。

4

2 回答 2

2

首先，您需要corpus reader为语料库制作一个。您可以在 中使用一些语料库阅读器nltk.corpus，例如：

AlpinoCorpusReader
BNCCorpusReader
BracketParseCorpusReader
CMUDictCorpusReader
CategorizedCorpusReader
CategorizedPlaintextCorpusReader
CategorizedTaggedCorpusReader
ChunkedCorpusReader
ConllChunkCorpusReader
ConllCorpusReader
CorpusReader
DependencyCorpusReader
EuroparlCorpusReader
IEERCorpusReader
IPIPANCorpusReader
IndianCorpusReader
MacMorphoCorpusReader
NPSChatCorpusReader
NombankCorpusReader
PPAttachmentCorpusReader
Pl196xCorpusReader
PlaintextCorpusReader
PortugueseCategorizedPlaintextCorpusReader
PropbankCorpusReader
RTECorpusReader
SensevalCorpusReader
SinicaTreebankCorpusReader
StringCategoryCorpusReader
SwadeshCorpusReader
SwitchboardCorpusReader
SyntaxCorpusReader
TaggedCorpusReader
TimitCorpusReader
ToolboxCorpusReader
VerbnetCorpusReader
WordListCorpusReader
WordNetCorpusReader
WordNetICCorpusReader
XMLCorpusReader
YCOECorpusReader

一旦你从你的语料库中制作了一个语料库阅读器，如下所示：

c = nltk.corpus.whateverCorpusReaderYouChoose(directoryWithCorpus, regexForFileTypes)

您可以使用以下代码从语料库中取出单词：

paragraphs = [para for para in c.paras()]
for para in paragraphs:
    words = [word for sentence in para for word in sentence]

这应该会为您提供语料库所有段落中所有单词的列表。

希望这可以帮助

于 2011-01-22T06:40:17.283 回答
1

您可以使用.words()nltk 语料库中的属性

content = nps_chat.words()

这将为您提供列表中的所有单词

['now', 'im', 'left', 'with', 'this', 'gay', 'name', ...]

于 2017-02-10T03:08:39.267 回答