2

我正在研究基于序列标记分类的 IOB 方案,

首先,我想读一下我的语料库和他们的标签,但是语料库已经以一种叫做 .ann 文件的格式保存,我在这里从来没有像你一样工作过。当我打开它时,它使用https://brat.nlplab.org/进行了注释 ,我看到了这个

T1  Claim 78 140    competition can effectively promote the development of economy
A1  Stance T1 Against
T2  MajorClaim 503 550  we should attach more importance to cooperation
T3  Premise 142 283 In order to survive in the competition, companies continue to improve their products and service, and as a result, the whole society prospers
T4  Claim 591 714   through cooperation, children can learn about interpersonal skills which are significant in the future life of all students
A2  Stance T4 For
T5  Premise 716 851 What we acquired from team work is not only how to achieve the same goal with others but more importantly, how to get along with others
T6  Premise 853 1086    During the process of cooperation, children can learn about how to listen to opinions of others, how to communicate with others, how to think comprehensively, and even how to compromise with other team members when conflicts occurred
T7  Premise 1088 1191   All of these skills help them to get on well with other people and will benefit them for the whole life
T8  Claim 1332 1376 competition makes the society more effective
A3  Stance T8 Against
T9  Premise 1212 1301   the significance of competition is that how to become more excellence to gain the victory
T10 Premise 1387 1492   when we consider about the question that how to win the game, we always find that we need the cooperation
T11 Premise 1549 1846   Take Olympic games which is a form of competition for instance, it is hard to imagine how an athlete could win the game without the training of his or her coach, and the help of other professional staffs such as the people who take care of his diet, and those who are in charge of the medical care
T12 Premise 1848 1915   The winner is the athlete but the success belongs to the whole team
T13 Claim 1927 1992 without the cooperation, there would be no victory of competition
A4  Stance T13 For
T14 Claim 2154 2231 a more cooperative attitudes towards life is more profitable in one's success
A5  Stance T14 For
R1  supports Arg1:T3 Arg2:T1    
R2  attacks Arg1:T1 Arg2:T2 
R3  supports Arg1:T5 Arg2:T4    
R4  supports Arg1:T6 Arg2:T4    
R5  supports Arg1:T7 Arg2:T4    
R6  supports Arg1:T9 Arg2:T8    
R7  supports Arg1:T11 Arg2:T12  
R8  supports Arg1:T12 Arg2:T13  
R9  supports Arg1:T10 Arg2:T13  
R10 supports Arg1:T4 Arg2:T2    
R11 attacks Arg1:T8 Arg2:T2 
R12 supports Arg1:T13 Arg2:T2   
R13 supports Arg1:T14 Arg2:T2   

我想轻松解码,并将我的数据保存为这种格式的数据框:

带有标签的句子(如您在文本中看到的那样,声明或前提或 MAJORCLAIM)

类似这种格式的东西

在此处输入图像描述

带有标签的句子

我尝试使用此功能读取 .txt 文件

myList = []                #read the whole text from 
for root, dirs, files in os.walk(path):
    for file in files:
        if file.endswith('.txt'):
            with open(os.path.join(root, file), 'r', encoding="utf-8") as f:
                text = f.read()
                myList.append(text)
df = pd.DataFrame(np.array(myList),index=list(range(1,len(myList)+1)),columns=["Paragraph"])

但是对于brat提供的这个ann文件,我就不知道了

4

3 回答 3

1

我不确定您究竟需要如何格式化此数据框,但无论您是否需要两列,都可以使用正则表达式分隔符来查找第一个空格,并在读入 pandas 数据框时将其用作分隔符。

df = pd.read_csv('test.ann', sep='^([^\s]*)\s', engine='python', header=None).drop(0, axis=1)

如果保存为 .ann 文件,这适用于您提供的上述示例。

于 2020-10-27T21:21:04.377 回答
0

似乎这是最好的方法

from brat_parser import get_entities_relations_attributes_groups

entities, relations, attributes, groups = get_entities_relations_attributes_groups("..\data\corpus02\essay01.ann")

使用这个方法我可以读取 .ann 文件!

{'T1': Entity(id='T1', type='Claim', span=((78, 140),), text='competition can effectively promote the development of economy'),
 'T2': Entity(id='T2', type='MajorClaim', span=((503, 550),), text='we should attach more importance to cooperation'),
 'T3': Entity(id='T3', type='Premise', span=((142, 283),), text='In order to survive in the competition, companies continue to improve their products and service, and as a result, the whole society prospers'),
 'T4': Entity(id='T4', type='Claim', span=((591, 714),), text='through cooperation, children can learn about interpersonal skills which are significant in the future life of all students'),
 'T5': Entity(id='T5', type='Premise', span=((716, 851),), text='What we acquired from team work is not only how to achieve the same goal with others but more importantly, how to get along with others'),
 'T6': Entity(id='T6', type='Premise', span=((853, 1086),), text='During the process of cooperation, children can learn about how to listen to opinions of others, how to communicate with others, how to think comprehensively, and even how to compromise with other team members when conflicts occurred'),
 'T7': Entity(id='T7', type='Premise', span=((1088, 1191),), text='All of these skills help them to get on well with other people and will benefit them for the whole life'),
 'T8': Entity(id='T8', type='Claim', span=((1332, 1376),), text='competition makes the society more effective'),
 'T9': Entity(id='T9', type='Premise', span=((1212, 1301),), text='the significance of competition is that how to become more excellence to gain the victory'),
 'T10': Entity(id='T10', type='Premise', span=((1387, 1492),), text='when we consider about the question that how to win the game, we always find that we need the cooperation'),
 'T11': Entity(id='T11', type='Premise', span=((1549, 1846),), text='Take Olympic games which is a form of competition for instance, it is hard to imagine how an athlete could win the game without the training of his or her coach, and the help of other professional staffs such as the people who take care of his diet, and those who are in charge of the medical care'),
 'T12': Entity(id='T12', type='Premise', span=((1848, 1915),), text='The winner is the athlete but the success belongs to the whole team'),
 'T13': Entity(id='T13', type='Claim', span=((1927, 1992),), text='without the cooperation, there would be no victory of competition'),
 'T14': Entity(id='T14', type='Claim', span=((2154, 2231),), text="a more cooperative attitudes towards life is more profitable in one's success")}

这是结果。将其转换为数据框应该不难

于 2020-10-27T22:03:35.857 回答
0

我的问题有这个答案

df = pd.read_csv('..\data\corpus02\essay01.ann', sep='^([^\s]*)\s', engine='python', header=None).drop(0, axis=1)

在此处输入图像描述

但这不是我想要的

我只想提取句子及其标签(Claim、Premise、Majorclaim),但在这里它分开 T1 或......我不想要它们

只有句子+他们的标签

于 2020-10-27T21:58:27.710 回答