1

我有一系列随机模拟的输出，格式为 .csv 文件，如下所示：

Run,ID,Var
1,1,7
1,2,9
1,3,4
2,1,3
2,2,4
2,3,8

等等

除此之外，我还有另一个数据文件，也是一个 .csv，格式如下：

ID, Var2, Var3
1,0.89,0.10
2,0.45,0.98
3,0.27,0.05
4,0.98,0.24

注意：数据文件中有一些值没有出现在模拟文件中。我希望这些被忽略。

我想做的是编写一个脚本，ID从第一个 .csv 文件中获取每个值，找到 Var2 和 Var3 并将它们放在一起，最终得到如下内容：

Run, ID, Var, Var2, Var3
1,1,7,0.89,0.10
1,2,9,0.45,0.98
1,3,4,0.27,0.05
2,1,3,0.89,0.10
2,2,4,0.45,0.98
2,3,8,0.27,0.05

关于如何做到这一点的任何建议？我承认这是我对 Python 中数据处理的理解的极限。我对如何在 SAS 中执行此操作有一定的了解，但我更愿意将其保留为单语言任务，以便可以将它们作为单个脚本处理。

4

3 回答 3

3

输出.csv：

Run, ID, Var
1, 1, 7
1, 2, 9
1, 3, 4
2, 1, 3
2, 2, 4
2, 3, 8

数据.csv：

ID, Var2, Var3
1, 0.89, 0.10
2, 0.45, 0.98
3, 0.27, 0.05
8, 0.4, 0.5

请注意，即使我们在 data.csv 中有条目，但不存在于 ouput.csv 中，它也不会影响最终结果，因为当我们解析 output.csv 时，我们只查找我们从 output.csv 中知道的 ID，尽管相反， data.csv 至少必须包含 output.csv 中的所有 ID，但如果需要，这可以很容易地处理。

代码：

import csv
from pprint import pprint 

data = dict([(row['ID'], row) for row in csv.DictReader(open('data.csv', 'rb'), skipinitialspace = True)])
values = []
for row in csv.DictReader(open('output.csv', 'rb'), skipinitialspace = True):
    values.append(row)
    values[-1].update(data[row['ID']])

>>> pprint(values)
[{'ID': '1', 'Run': '1', 'Var': '7', 'Var2': '0.89', 'Var3': '0.10'},
 {'ID': '2', 'Run': '1', 'Var': '9', 'Var2': '0.45', 'Var3': '0.98'},
 {'ID': '3', 'Run': '1', 'Var': '4', 'Var2': '0.27', 'Var3': '0.05'},
 {'ID': '1', 'Run': '2', 'Var': '3', 'Var2': '0.89', 'Var3': '0.10'},
 {'ID': '2', 'Run': '2', 'Var': '4', 'Var2': '0.45', 'Var3': '0.98'},
 {'ID': '3', 'Run': '2', 'Var': '8', 'Var2': '0.27', 'Var3': '0.05'}]
>>>

现在保存回 csv 文件。

fieldnames = ['Run', 'ID', 'Var', 'Var2', 'Var3']
f = open('combined.csv', 'wb')
csvwriter = csv.DictWriter(f, fieldnames = fieldnames)
csvwriter.writerow(dict((fn,fn) for fn in fieldnames)) # 2.7 has writeheader, which is cleaner
[csvwriter.writerow(row) for row in values]
f.close()


$ cat combined.csv 
Run,ID,Var,Var2,Var3
1,1,7,0.89,0.10
1,2,9,0.45,0.98
1,3,4,0.27,0.05
2,1,3,0.89,0.10
2,2,4,0.45,0.98
2,3,8,0.27,0.05

我希望这有帮助。

于 2012-07-07T08:20:18.873 回答
1

不使用csv模块的解决方案：

with open('data.txt') as f1,open('data1.txt') as f2,open('data3.txt','w') as f3:
    header1=f1.readline().strip().split(',') #header from file 1 i.e Run,ID,Var

    header2=f2.readline().strip().split(',')[1:] #header from file 2 ,i.e Var2, Var3

    dic={x.strip().split(',')[0]:x.strip().split(',')[1:] for x in f2 if x.strip()} #use dict to save data as per ID from file 2

    f3.write(','.join((header1+header2))+'\n') #write the new header(header1+header2) to file 3

    for x in f1: 
        f3.write(x.strip()+','+','.join(dic[x.split(',')[1]])+'\n') #fetch results from dic as per the ID obtained from the current line in data.txt

输出： data3.txt包含

Run,ID,Var, Var2, Var3
1,1,7,0.89,0.10
1,2,9,0.45,0.98
1,3,4,0.27,0.05
2,1,3,0.89,0.10
2,2,4,0.45,0.98
2,3,8,0.27,0.05
于 2012-07-07T08:04:54.193 回答
0

简单易行：

f = open('one.csv', 'r')
one = f.read()
f.close()

f = open('two.csv', 'r')
two = f.read()
f.close()

one = one.split('\n')[1:-1]
two = two.split('\n')[1:-1]
output = 'Run, ID, Var, Var2, Var3\n'

for o in one:
  for t in two:
    row = t.split(',')
    if o.split(',')[1] == row[0]:
      output += '%s,%s,%s\n' % (o, row[1], row[2])

# or save it to a file
print output
于 2012-07-07T08:47:26.897 回答