1

Suppose I have the following txt file:

0.0163934
6
7.52438e+09
2147483648
6.3002e-06 6.31527e-08 0 0 6 0 0 4.68498e-06 0.00638412 12.6688
6.33438e-06 0 5.99588e-09 0 0 0 0 4.70195e-06 0 12.876
6.36874e-06 0 6.09398e-09 0 0 0 0 4.71894e-06 0 13.0867
6.40329e-06 0 6.19369e-09 0 0 0 0 4.73593e-06 0 13.3009
6.43802e-06 0 6.29503e-09 0 0 0 0 4.75294e-06 0 13.5185
6.47295e-06 0 6.39803e-09 0 0 0 0 4.76996e-06 0 13.7397
0.0163934
3
7.52438e+09
2147483648
6.3002e-06 0 5.89935e-09 0 0 0 0 4.68498e-06 0 12.6688
6.33438e-06 0 5.99588e-09 0 0 0 0 4.70195e-06 0 12.876
6.36874e-06 0 6.09398e-09 0 0 0 0 4.71894e-06 0 13.0867

I want to read each of the first lines as floats or integers and then depending on the second line I want to read the rest of lines as a list of lists or array.

In IDL language I just have to do: 

openr, 1, fname
readf, 1, Time
readf, 1, Bins
readf, 1, dummy
readf, 1, dummyLong
da1= fltarr(10, Bins)
readf, 1, da1

So that the entire block of numbers is stored in the integer da1 which is size: 10*Bins. (rows and columns are the opposite as in python)

And then I can read the following lines in the same way.

In python I am doing: 

Time=float(filen.readline())
Bins=int(filen.readline())
dummy=float(filen.readline())
dummyLong=long(filen.readline())

lines=[filen.readline() for i in range(Bins)]

arra=[[float(x) for x in lines[i].split()] for i in range(len(lines))]

So I need two lines of code and complicated iterations that are not understandable to a beginner.

Is there a way to do it like in IDL, in a single statement and pythonic?

Thanks!

4

4 回答 4

1

这是一种更面向对象的方法，使用简单编码的 FSM（有限状态机）来控制读取完整数据记录的过程。当前发布的其他答案更为冗长，但它是一种相当灵活且可扩展的方式来处理此类任务并通过错误检查来完成。

class Record(object):
    def __init__(self, time=None, bins=None, fltarr=None):
        self.time = time
        self.bins = bins
        self.fltarr = fltarr

    def read(self, file):
        """ Read complete record from file into self and return True,
            otherwise return False if EOF encountered """
        START, STOP, EOF = 0, -1, -99

        state = START
        while state not in (EOF, STOP):
            line = file.readline()
            if not line: state = EOF; break
            # process line depending on read state
            if state == 0:
                self.time = float(line)
                state = 1
            elif state == 1:
                self.bins = int(line)
                state = 2
            elif state in (2, 3):
                # ignore line
                state += 1
            elif state == 4:
                self.fltarr = []
                last_bin = self.bins-1
                for bin in xrange(self.bins):
                    self.fltarr.append([float(x) for x in line.split()])
                    if bin == last_bin: break
                    line = file.readline()
                    if not line: state = EOF; break
                if state != EOF:
                    state = STOP

        return state == STOP

    def __str__(self):
        result = 'Record(time={}, bins={}, fltarr=[\n'.format(self.time, self.bins)
        for floats in self.fltarr:
            result += '  {}\n'.format(floats)
        return result + '])'

fname = 'sample_data.txt'
with open(fname, 'r') as input:
    data = []
    while True:
        record = Record()
        if not record.read(input):
            break
        else:
            data.append(record)

for record in data:
    print record

输出：

Record(time=0.0163934, bins=6, fltarr=[
  [6.3002e-06, 6.31527e-08, 0.0, 0.0, 6.0, 0.0, 0.0, 4.68498e-06, 0.00638412, 12.6688]
  [6.33438e-06, 0.0, 5.99588e-09, 0.0, 0.0, 0.0, 0.0, 4.70195e-06, 0.0, 12.876]
  [6.36874e-06, 0.0, 6.09398e-09, 0.0, 0.0, 0.0, 0.0, 4.71894e-06, 0.0, 13.0867]
  [6.40329e-06, 0.0, 6.19369e-09, 0.0, 0.0, 0.0, 0.0, 4.73593e-06, 0.0, 13.3009]
  [6.43802e-06, 0.0, 6.29503e-09, 0.0, 0.0, 0.0, 0.0, 4.75294e-06, 0.0, 13.5185]
  [6.47295e-06, 0.0, 6.39803e-09, 0.0, 0.0, 0.0, 0.0, 4.76996e-06, 0.0, 13.7397]
])
Record(time=0.0163934, bins=3, fltarr=[
  [6.3002e-06, 0.0, 5.89935e-09, 0.0, 0.0, 0.0, 0.0, 4.68498e-06, 0.0, 12.6688]
  [6.33438e-06, 0.0, 5.99588e-09, 0.0, 0.0, 0.0, 0.0, 4.70195e-06, 0.0, 12.876]
  [6.36874e-06, 0.0, 6.09398e-09, 0.0, 0.0, 0.0, 0.0, 4.71894e-06, 0.0, 13.0867]
])
于 2013-02-01T19:05:25.927 回答
1

一条线不一定比两条线好。

但你可以这样做：

arra = [[float(x) for x in filen.readline().split()] for _ in range(Bins)]

我更喜欢两行：

lines = (filen.readline() for _ in range(Bins))
arra = [[float(x) for x in line.split()] for line in lines]
于 2013-02-01T16:34:13.807 回答
1 
Time=float(fname.readline())
Bins=int(fname.readline())
dummy=float(fname.readline())
dummyLong=long(fname.readline())
arra = [ [ float(num) for num in line.split() ] for line in filen ]

这只是稍微有点 Pythonic，但它不会在所需的行数之后停止读取，它只是将它们全部读取。您可以使用islicefromitertools来停止迭代，或者您可以在之后简单地截断列表。

这是一个例子，因为我已经在使用了，所以我islice冒昧地对函数式编程感兴趣......

from itertools import islice

CONVERTORS = (float, int, float, long, )
with open(...) as filen:
    Time, Bins, dummy, dummyLong = [ func(value) for func, value in zip(CONVERTORS, islice(filen, 4)) ]
    arra = [ map(float, line.split()) for line in islice(filen, Bins) ]
于 2013-02-01T16:36:48.167 回答
0

您还可以使用 numpy loadtxt 之类的

from numpy import loadtxt
data = loadtxt("input.txt", unpack=False)

然后根据需要转换数据类型

或者，也可以使用 readlines：

from numpy import fromstring
fin = open("filename.dat")
data = fin.readlines()
Bins = -3
for record range(no_of_records):
    i = record + 3 + Bins
    Time = float(data[i])
    Bins = int(data[i+1])
    dummy, dummylong = (float(data[i+2]),float(data[i+3]))
    Bins = [fromstring(data(i+4+j), dtype=float, sep=" ") for j in range(Bins)]
于 2013-02-04T11:00:19.927 回答