Splitting up a data file in Python, round 2

Question

So I have recently found a solution to this question here in which I wanted to take two columns of a data file and put them into two arrays I now have this code which does the job nicely.

Xvals=[]; Yvals=[]
i = open('BGBiasRE_IM3_wCSrescaled.txt','r')

lines = [line.split() for line in i if line[:4] not in ('time', 'Step')]
Xvals, Yvals = zip(*lines)

V = [0, 0.004, 0, 0.0004]

pylab.plot(Xvals, Yvals, marker='o')
pylab.axis(V)
pylab.xlabel('Time (ms)')
pylab.ylabel('Current (A)')
pylab.title('Title')
pylab.show()

But I now realise I screwed up the question. I have a data file laid out as below,

    time    I(R_stkb)
Step Information: Temp=0  (Run: 1/11)

0.000000000000000e+000  0.000000e+000

9.999999960041972e-012  8.924141e-012

1.999999992008394e-011  9.623148e-012

Step Information: Temp=10  (Run: 2/11)

0.000000000000000e+000  0.000000e+000

9.999999960041972e-012  4.924141e-012

1.999999992008394e-011  8.623148e-012

(Note: No empty lines between each data line, and a Tab between the two data values)

The above code appends all the step information into one array, so I get two big long arrays when I want two different arrays for the different steps so I can plot their respective arrays separately later on. I also have to get the Step name, in this case Temp=10 and attach it/name the array to reflect each chunk of step info. Eg. I would like to end up with arrays like such

Temp0_Xvals = [ 0.000000000000000e+000, 9.999999960041972e-012, 1.999999992008394e-011]
Temp0_Yvals = [ 0.000000e+000, 8.924141e-012, 9.623148e-012]

Temp10_Xvals = [...]
Temp10_Yvals = [...] etc etc

Obviously this makes the problem much more complicated and I have no idea where to start.

Answer 1

I would do something along those lines:

i = open('BGBiasRE_IM3_wCSrescaled.txt', 'r')

Xnames, Ynames = [], []

count = 0
for line in i:
    if count > 0:
        line_tmp = line.split()
        if line_tmp[0] == 'Step':
            step = (line_tmp[2].split('='))[1]
            varnameX = 'Temp' + str(step) +'_Xvals'
            varnameY = 'Temp' + str(step) +'_Yvals'
            globals()[varnameX] = []
            globals()[varnameY] = []
            Xnames.append(varnameX)
            Ynames.append(varnameY)
        else:
            globals()[varnameX].append(float(line_tmp[0]))
            globals()[varnameY].append(float(line_tmp[1]))
    count += 1
i.close()

for name in Xnames:
    print name + ' = ' + str(eval(name))

for name in Ynames:
    print name + ' = ' + str(eval(name))

This is for sure not the most efficient solution but it works for your specific problem

Answer 2

Use csv with the excel_tab dialect.