5

我的一个脚本有问题……我会把问题放在项目符号上。

  • 问题/问题 1 - 在保存之前将原始 testing.csv 与修改后的进行比较，如果不同则应保存，如果相同则不应保存。
    • 在我下面的代码中，数据是相同的，但由于某种原因它认为它不同，我不知道为什么......
  • 问题/问题 2 - 在比赛期间忽略某些数据
    • 我想匹配使用MATCH2但忽略括号数据，例如在最后一个类数据MATCH2中Mdata (D)它需要匹配MData
  • 问题/问题 3 - 切片数据以匹配
    • 我想找到一种方法，所以如果我想使用MATCH1我可以设置MATCH1它只使用MATCH1[-1:]最终会给我这个例子中的数字。

Testing.py

import re
import pandas
from pandas.util.testing import assert_frame_equal

# each block in the text file will be one element of this list
matchers = [[]]
i = 0 
with open('testing.txt') as infile:
    for line in infile:
        line = line.strip()
        # Blocks are seperated by blank lines
        if len(line) == 0:
            i += 1
            matchers.append([])
            # assume there are always two blank lines between items 
            # and just skip to the lext line
            infile.next()
            continue
        matchers[i].append(line)


# This regular expression matches the variable number of students in each block
studentlike = re.compile('(\d+) (.+) (\d+/\d+)')
# These are the names of the fields we expect at the end of each block
datanames = ['Data', 'misc2', 'bla3']
# We will build a table containing a list of elements for each student
table = []
for matcher in matchers:
    # We use an iterator over the block lines to make indexing simpler
    it = iter(matcher)
    # The first two elements are match values
    m1, m2 = it.next(), it.next()
    # then there are a number of students
    students = []
    for possiblestudent in it:
        m = studentlike.match(possiblestudent)
        if m:
            students.append(list(m.groups()))
        else:
            break
    # After the students come the data elements, which we read into a dictionary
    # We also add in the last possible student line as that didn't match the student re
    dataitems = dict(item.split() for item in [possiblestudent] + list(it))
    # Finally we construct the table
    for student in students:
        # We use the dictionary .get() method to return blanks for the missing fields
        table.append([m1, m2] + student + [dataitems.get(d, '') for d in datanames])

textcols = ['MATCH2', 'MATCH1', 'TITLE01', 'MATCH3','TITLE02', 'Data', 'misc2', 'bla3']
csvdata = pandas.read_csv('testing.csv')
csvdata_old = csvdata.copy()
textdata = pandas.DataFrame(table, columns=textcols)

# Add any new columns
newCols = textdata.columns - csvdata.columns
for c in newCols:
    csvdata[c] = None

mergecols = ['MATCH2', 'MATCH1', 'MATCH3']
csvdata.set_index(mergecols, inplace=True, drop=False)
csvdata_old.set_index(mergecols, inplace=True, drop=False)
textdata.set_index(mergecols, inplace=True,drop=False)

csvdata.update(textdata)

try:
    assert_frame_equal(csvdata, csvdata_old)
    print "True (Same)"
except:
    csvdata.to_csv('testing.csv', index=False)
    print "False (Different)"

testing.txt

MData
DMATCH1
3 Tommy 144512/23332
1 Jim 90000/222311
1 Elz M 90000/222311
1 Ben 90000/222311
Data $50.90
misc2 $10.40
bla3 $20.20


MData
DMATCH2
4 James Smith 2333/114441
4 Mike 90000/222311
4 Jessica Long 2333/114441
Data $50.90
bla3 $5.44


Mdata
DMATCH3
5 Joe Reane 0/0
5 Peter Jones 90000/222311
Data $10.91
misc2 $420.00
bla3 $210.00

testing.csv

MATCH1,MATCH2,TITLE,TITLE,TITLE,TITLE,TITLE,TITLE,MATCH3,DATA,TITLE,TITLE
DMATCH1,MData (N/A),data,data,data,data,data,data,Tommy,55,data,data
DMATCH1,MData (N/A),data,data,data,data,data,data,Ben,54,data,data
DMATCH1,MData (N/A),data,data,data,data,data,data,Jim,52,data,data
DMATCH1,MData (N/A),data,data,data,data,data,data,Elz M,22,data,data
DMATCH2,MData (B/B),data,data,data,data,data,data,James Smith,15,data,data
DMATCH2,MData (B/B),data,data,data,data,data,data,Jessica Long,224,data,data
DMATCH2,MData (B/B),data,data,data,data,data,data,Mike,62,data,data
DMATCH3,Mdata (D),data,data,data,data,data,data,Joe Reane,66,data,data
DMATCH3,Mdata (D),data,data,data,data,data,data,Peter Jones,256,data,data
DMATCH3,Mdata (D),data,data,data,data,data,data,Lesley Lope,5226,data,data

脚本testing.csv运行后需要...

MATCH1,MATCH2,TITLE,TITLE.1,TITLE.2,TITLE.3,TITLE.4,TITLE.5,MATCH3,DATA,TITLE.6,TITLE.7,Data,TITLE01,TITLE02,bla3,misc2
DMATCH1,MData (N/A),data,data,data,data,data,data,Tommy,55,data,data,$50.90,3,144512/23332,$20.20,$10.40
DMATCH1,MData (N/A),data,data,data,data,data,data,Ben,54,data,data,$50.90,1,90000/222311,$20.20,$10.40
DMATCH1,MData (N/A),data,data,data,data,data,data,Jim,52,data,data,$50.90,1,90000/222311,$20.20,$10.40
DMATCH1,MData (N/A),data,data,data,data,data,data,Elz M,22,data,data,$50.90,1,90000/222311,$20.20,$10.40
DMATCH2,MData (B/B),data,data,data,data,data,data,James Smith,15,data,data,$50.90,4,2333/114441,$5.44,
DMATCH2,MData (B/B),data,data,data,data,data,data,Jessica Long,224,data,data,$50.90,4,2333/114441,$5.44,
DMATCH2,MData (B/B),data,data,data,data,data,data,Mike,62,data,data,$50.90,4,90000/222311,$5.44,
DMATCH3,Mdata (D),data,data,data,data,data,data,Joe Reane,66,data,data,$10.91,5,0/0,$210.00,$420.00
DMATCH3,Mdata (D),data,data,data,data,data,data,Peter Jones,256,data,data,$10.91,5,90000/222311,$210.00,$420.00
DMATCH3,Mdata (D),data,data,data,data,data,data,Lesley Lope,5226,data,data,,,,,

如果有人可以，我将非常感谢您的帮助:)

为 bheklilr 编辑

testing.txt

Mdata
DMATCH3
5 Joe Reane 0/0
5 Peter Jones 90000/222311
Data $10.91
misc2 $420.00
bla3 $210.00

testing.csv

MATCH1,MATCH2,TITLE,MATCH3,DATA,TITLE
DMATCH3,Mdata (D),data,Joe Reane,66,data
DMATCH3,Mdata (D),data,Peter Jones,256,data
DMATCH3,Mdata (D),data,Lesley Lope,5226,data

脚本testing.csv运行后需要...

MATCH1,MATCH2,TITLE,MATCH3,DATA,TITLE.1,Data,TITLE01,TITLE02,bla3,misc2
DMATCH3,Mdata (D),data,Joe Reane,66,data,$10.91,5,0/0,$210.00,$420.00
DMATCH3,Mdata (D),data,Peter Jones,256,data,$10.91,5,90000/222311,$210.00,$420.00
DMATCH3,Mdata (D),data,Lesley Lope,5226,data,,,,,
4

1 回答 1

0

通过您的示例，这里有一些答案：

问题 1

在保存之前将原始 testing.csv 与修改后的进行比较，如果不同则应保存，如果相同则不应保存。

答案 1 - 它们实际上是不同的。例如，让我们在脚本末尾输出每个 csv 的样子：

# This is near the end of your script...

mergecols = ['MATCH2', 'MATCH1', 'MATCH3']
csvdata.set_index(mergecols, inplace=True, drop=False)
csvdata_old.set_index(mergecols, inplace=True, drop=False)
textdata.set_index(mergecols, inplace=True,drop=False)

csvdata.update(textdata)

# Add these 3 lines to dump your files to csv's of the same name.
csvdata.to_csv('csvdata.csv')
csvdata_old.to_csv('csvdata_old.csv')
textdata.to_csv('textdata.csv')

try:
    assert_frame_equal(csvdata, csvdata_old)
    print "True (Same)"
except:
    print "False (Different)"

csvdata在文本编辑器或 excel 中检查生成的 csv 文件，您会看到它csvdata_old实际上是不同的。他们为什么不呢。您已经使用这部分 python 代码向其中添加了列：

>>> textdata.columns - csvdata.columns
Index([Data, TITLE01, TITLE02, bla3, misc2], dtype=object)

所以，你的断言是正确的。他们是不同的。

我不太清楚你在问题 2 或 3 中所说的“匹配”是什么意思。你想匹配什么？您实际上是指按类似列分组结果吗？这看起来如何？更多信息会很棒！

于 2013-11-16T02:52:46.883 回答