Python reading X and Y values and assigning to Z specific X Y pairs

Question

I have used python to create 2D XY plots in the past from pairs of data in a file, but now I need to create a contour plot from data I have in a file. The file looks like this:

<Descriptive string>
<some "random" number>
<number of X values:nx>
<Number of Y values:ny>
X1 X2 X3 X4 X5
X6 X7 X8 X9 X10
...
... Xnx
Y1 Y2 Y3 Y4 Y5
Y6 Y7 Y8 Y9 Y10
...
... Yny
Z(X1,Y1) Z(X1,Y2) Z(X1,Y3) Z(X1,Y4) Z(X1,Y5)
Z(X1,Y6) Z(X1,Y7) Z(X1,Y8) Z(X1,Y9) Z(X1,Y10)
...
... 
Z(X1,Yny) Z(X2,Y1) Z(X2,Y3) Z(X2,Y4) Z(X2,Y5)
...
...
Z(X2,Yny) ...
...
...
Z(Xnx,Yny)

So far I've been able to read the values of X and Y although probably not in the most convenient form, even the values of Z, but then I am unable to assign them properly so that Z1 goes with (X1,Y1), Z2 with (X1,Y2) up to Zny going with (X1,Yny) and at the end Znx*ny going with (Xnx,Yny). Hope this is clear enough... So far, this is the piece of code that I have:

import numpy as np

# Read from .dat file:
with open("trans_acrolein_ResWVFunAP1R12.dat", "r") as f:
    fl = f.readline()
    xnum = f.readlines()[2]
    ynum = f.readlines()[3]

# Initialize some variables to be lists.
xval = []
yval = []

# Read the values of the number of X and Y values.
for line in xnum:
    px = line.split()
    xval.append(int(px[0]))

for line in ynum:
    py = line.split()
    yval.append(int(py[0]))

linesx = np.ceil(xval/5.0)
linesy = np.ceil(yval/5.0)
linesz = np.ceil((xval*yval)/5.0)

with open("trans_acrolein_ResWVFunAP1R12.dat", "r") as f:
    for line in f:
        x = []
        y = []
        for element in line[4:3+linesx].split():
            x.append(element)
            for element in line[4+linesx:3+linesx+linesy].split():
                y.append(element)

Since I don't know how to read all the elements consecutively I used first a way of counting the number of lines that have X values, then the ones with Y values, and finally the ones with Z values, but I guess its not very efficient. I will really appreciate if someone can help me with this. Thanks,

Answer 1

I assume that N_z_values = N_x_values * N_y_values. If there are the same number of values on each line, then you should be able to parse all the data containing lines at once, then split it up according to N_x_values.

For example, if N_x_values = 27 and N_y_values = 28 and there are 5 values per line, then you could do

import numpy as np
data = []
N_x_values, N_y_values = 0, 0
with open(file_name, 'r') as in_file:
    # skip 2 lines, grab N_X, grab N_Y
    [in_file.next() for _ in range(2)]
    N_x_values = int(in_file.next().strip())
    N_y_values = int(in_file.next().strip())

    for line in in_file:
        line = line.strip().split(' ')
        data.append(map(float, line))

data = np.array(data)
data = data.reshape(np.prod(data.shape))
x_cutoff = N_x_values
y_cutoff = N_y_values
x = data[:x_cutoff]
y = data[x_cutoff:y_cutoff]
z = data[y_cutoff:].reshape(N_x_values, N_y_values)

Now your values are in arrays of the form

x.shape = (27, )
y.shape = (28, )
z.shape = (27, 28)

matplotlib.pyplot.contour can take these values directly. For example

import matplotlib.pyplot as plt
f = plt.figure()
ax = f.add_suplot(111)
ax.contourf(x, y, z)
ax.colorbar()
plt.show()