Open with 'rU' not recognizing endlines

Question

I have a text file created by a Fortran program ran on SL6. It looks like:

1

R - Z     binning n.   1  "1         " , generalized particle n.  223      
  R coordinate: from  0.0000E+00 to  1.1000E+02 cm,   110 bins ( 1.0000E+00 cm wide)
  Z coordinate: from -2.9000E+02 to  2.9000E+02 cm,   290 bins ( 2.0000E+00 cm wide)
  axis coordinates: X = 0.0000E+00, Y =  0.0000E+00 cm
  Data follow in a matrix A(ir,iz), format (1(5x,1p,10(1x,e11.4)))

 accurate deposition along the tracks requested
  this is a track-length binning
   3.0406E-01  2.3565E-02  1.0664E-02  7.2081E-03  5.2534E-03  4.8756E-03  4.5011E-03  4.2792E-03  4.1801E-03  3.9648E-03
   3.9108E-03  3.8301E-03  3.7256E-03  3.6330E-03  3.5912E-03  3.5461E-03  3.4579E-03  3.4813E-03  3.4395E-03  3.3868E-03
   And so on for 6000 lines...

I want to read all the numbers into a list of lists, so I have to skip the first nine lines, but Python is not recognizing the endlines despite opening as 'rU'. Just as a test, this code:

f = open(file, 'rU')
print f.readlines(2)

Outputs (with '\n's read as part of the string):

BlBlockq['1\n', ' R - Z binning n. 1 "1 " , generalized particle n. 223\n', ' R coordinate: from 0.0000E+00 to 1.1000E+02 cm, 110 bins ( 1.0000E+00 cm wide)\n', ' Z coordinate: from -2.9000E+02 to 2.9000E+02 cm, 290 bins ( 2.0000E+00 cm wide)\n', ' axis coordinates: X = 0.0000E+00, Y = 0.0000E+00 cm\n', ' Data follow in a matrix A(ir,iz), format (1(5x,1p,10(1x,e11.4)))\n', '\n', ' accurate deposition along the tracks requested\n', ' this is a track-length binning\n', ' 3.0406E-01 2.3565E-02 1.0664E-02 7.2081E-03 5.2534E-03 4.8756E-03 4.5011E-03 4.2792E-03 4.1801E-03 3.9648E-03 \n', ' 3.9108E-03 3.8301E-03 3.7256E-03 3.6330E-03 3.5912E-03 3.5461E-03 3.4579E-03 3.4813E-03 3.4395E-03 3.3868E-03\n', ' 3.3292E-03 3.2912E-03 3.2342E-03 3.1778E-03 3.1790E-03 3.1501E-03 3.1095E-03 3.0531E-03 3.0162E-03 3.0427E-03\n', ' 2.9452E-03 2.8939E-03 2.8759E-03 2.8347E-03 2.8078E-03 2.7564E-03 2.7169E-03 2.7287E-03 2.6690E-03 2.6258E-03\n', ' 2.6070E-03 2.5763E-03 2.5385E-03 2.5521E-03 2.4891E-03 2.4825E-03 2.4690E-03 2.4200E-03 2.3839E-03 2.3437E-03\n', ' 2.3140E-03 2.3068E-03 2.2621E-03 2.2337E-03 2.2014E-03 2.1855E-03 2.1596E-03 2.1531E-03 2.1182E-03 2.1182E-03\n', ' 2.0879E-03 2.0681E-03 2.0307E-03 2.0172E-03 2.0099E-03 1.9916E-03 1.9581E-03 1.9515E-03 1.9164E-03 1.8902E-03\n', ' 1.8913E-03 1.8793E-03 1.8706E-03 1.8478E-03 1.8104E-03 1.8075E-03 1.7837E-03 1.7434E-03 1.7204E-03 1.7091E-03\n', ' 1.6946E-03 1.6776E-03 1.6521E-03 1.6512E-03 1.6228E-03 1.6121E-03 1.5918E-03 1.5848E-03 1.5748E-03 1.5444E-03\n', ' 1.5317E-03 1.5280E-03 1.5066E-03 1.4802E-03 1.4487E-03 1.4437E-03 1.4265E-03 1.4193E-03 1.4142E-03 1.3982E-03\n', ' 1.3899E-03 1.3632E-03 1.3435E-03 1.3263E-03 1.3241E-03 1.3130E-03 1.2938E-03 1.2828E-03 1.2769E-03 1.2656E-03\n', ' 3.0497E-01 2.3320E-02 1.0399E-02 7.1513E-03 5.2022E-03 4.8103E-03 4.4078E-03 4.1423E-03 4.0100E-03 3.8307E-03\n', ' 3.7849E-03 3.8155E-03 3.6134E-03 3.5784E-03 3.4755E-03 3.3726E-03 3.3375E-03 3.2878E-03 3.2996E-03 3.2516E-03\n', ' 3.2164E-03 3.1519E-03 3.1368E-03 3.1073E-03 3.0511E-03 3.0447E-03 3.0236E-03 2.9774E-03 2.9112E-03 2.8960E-03\n', ' 2.8631E-03 2.8311E-03 2.7943E-03 2.7737E-03 2.7331E-03 2.7091E-03 2.7088E-03 2.6326E-03 2.6396E-03 2.5996E-03\n', ' 2.5588E-03 2.5404E-03 2.5130E-03 2.4893E-03 2.4464E-03 2.4332E-03 2.4431E-03 2.3831E-03 2.3552E-03 2.3305E-03\n', ' 2.3033E-03 2.3048E-03 2.2519E-03 2.2359E-03 2.2127E-03 2.1836E-03 2.1486E-03 2.1193E-03 2.1092E-03 2.0924E-03\n', ' 2.0966E-03 2.0535E-03 2.0043E-03 2.0146E-03 1.9817E-03 1.9776E-03 1.9542E-03 1.9027E-03 1.8876E-03 1.8756E-03\n', ' 1.8760E-03 1.8721E-03 1.8438E-03 1.8270E-03 1.7769E-03 1.7644E-03 1.7371E-03 1.7354E-03 1.7224E-03 1.7026E-03\n', ' 1.6694E-03 1.6437E-03 1.6364E-03 1.6486E-03 1.6130E-03 1.6150E-03 1.6029E-03 1.5856E-03 1.5595E-03 1.5353E-03\n', ' 1.5230E-03 1.5227E-03 1.5091E-03 1.4668E-03 1.4580E-03 1.4477E-03 1.4292E-03 1.4027E-03 1.3987E-03 1.3952E-03\n', ' 1.3774E-03 1.3677E-03 1.3548E-03 1.3280E-03 1.3174E-03 1.3040E-03 1.2928E-03 1.2865E-03 1.2802E-03 1.2549E-03\n', ' 3.0509E-01 2.3272E-02 1.0357E-02 7.0505E-03 5.0352E-03 4.6982E-03 4.3180E-03 4.0935E-03 3.8469E-03 3.6525E-03\n', ' 3.5883E-03 3.5556E-03 3.4086E-03 3.4375E-03 3.3495E-03 3.2696E-03 3.2182E-03 3.2249E-03 3.0992E-03 3.0897E-03\n', ' 3.0598E-03 3.0394E-03 3.0273E-03 2.9569E-03 2.9956E-03 2.9403E-03 2.8916E-03 2.9049E-03 2.8632E-03 2.7784E-03\n', ' 2.7787E-03 2.7386E-03 2.7034E-03 2.6822E-03 2.6373E-03 2.6070E-03 2.5835E-03 2.5861E-03 2.5695E-03 2.5765E-03\n', ' 2.5236E-03 2.4904E-03 2.4510E-03 2.4232E-03 2.4039E-03 2.3812E-03 2.3566E-03 2.3374E-03 2.3185E-03 2.2919E-03\n', ' 2.2724E-03 2.2345E-03 2.2360E-03 2.2162E-03 2.1863E-03 2.1568E-03 2.1213E-03 2.1009E-03 2.0799E-03 2.0681E-03\n', ' 2.0659E-03 2.0498E-03 2.0391E-03 1.9898E-03 1.9438E-03 1.9394E-03 1.9185E-03 1.8904E-03 1.8929E-03 1.8697E-03\n', ' 1.8682E-03 1.8425E-03 1.8035E-03 1.7709E-03 1.7639E-03 1.7456E-03 1.7236E-03 1.7238E-03 1.7163E-03 1.7145E-03\n', ' 1.6621E-03 1.6467E-03 1.6545E-03 1.6298E-03 1.6022E-03 1.5985E-03 1.5753E-03 1.5643E-03 1.5375E-03 1.5466E-03\n', ' 1.5277E-03 1.5178E-03 1.4924E-03 1.4771E-03 1.4579E-03 1.4398E-03 1.4112E-03 1.4110E-03 1.3973E-03 1.3907E-03\n', ' 1.3802E-03 1.3493E-03 1.3278E-03 1.3369E-03 1.3274E-03 1.3164E-03 1.3124E-03 1.2916E-03 1.2747E-03 1.2640E-03\n', ' 3.0610E-01 2.3126E-02 1.0185E-02 6.8339E-03 4.8459E-03 4.5390E-03 4.1334E-03 3.9815E-03 3.8058E-03 3.5676E-03\n', ' 3.4392E-03 3.3838E-03 3.3611E-03 3.2674E-03 3.1803E-03 3.1474E-03 3.1800E-03 3.1263E-03 3.0985E-03 3.0649E-03\n', ' 3.0075E-03 2.9502E-03 2.9607E-03 2.9350E-03 2.9096E-03 2.8821E-03 2.8311E-03 2.7796E-03 2.7583E-03 2.7611E-03\n', ' 2.7295E-03 2.6707E-03 2.6291E-03 2.6345E-03 2.6062E-03 2.5963E-03 2.5672E-03 2.5447E-03 2.4997E-03 2.4905E-03\n', ' 2.4835E-03 2.4659E-03 2.3853E-03 2.3637E-03 2.3618E-03 2.3600E-03 2.3076E-03 2.2975E-03 2.2652E-03 2.2580E-03\n', ' 2.2591E-03 2.2327E-03 2.2344E-03 2.2002E-03 2.1638E-03 2.1359E-03 2.0953E-03 2.0861E-03 2.0702E-03 2.0468E-03\n', ' 2.0228E-03 1.9948E-03 1.9968E-03 1.9793E-03 1.9369E-03 1.9242E-03 1.8957E-03 1.8783E-03 1.8986E-03 1.8529E-03\n', ' 1.8327E-03 1.7979E-03 1.7969E-03 1.7697E-03 1.7726E-03 1.7304E-03 1.7385E-03 1.7116E-03 1.6935E-03 1.6875E-03\n', ' 1.6572E-03 1.6438E-03 1.6302E-03 1.6136E-03 1.5967E-03 1.5825E-03 1.5641E-03 1.5628E-03 1.5536E-03 1.5403E-03\n', ' 1.5359E-03 1.5081E-03 1.4791E-03 1.4639E-03 1.4470E-03 1.4157E-03 1.4050E-03 1.4019E-03 1.3899E-03 1.3760E-03\n', ' 1.3686E-03 1.3614E-03 1.3509E-03 1.3426E-03 1.3354E-03 1.3160E-03 1.3000E-03 1.2933E-03 1.2744E-03 1.2754E-03\n', ' 3.0636E-01 2.3067E-02 1.0104E-02 6.7223E-03 4.7879E-03 4.4974E-03 3.9843E-03 3.7891E-03 3.5955E-03 3.5142E-03\n', ' 3.3559E-03 3.3439E-03 3.2640E-03 3.2311E-03 3.1302E-03 3.0504E-03 3.0634E-03 3.0229E-03 3.0359E-03 2.9668E-03\n', ' 2.9471E-03 2.9103E-03 2.8339E-03 2.8224E-03 2.8151E-03 2.7428E-03 2.7436E-03 2.7491E-03 2.7130E-03 2.6586E-03\n', ' 2.6498E-03 2.6161E-03 2.5706E-03 2.5646E-03 2.5351E-03 2.4974E-03 2.4919E-03 2.4849E-03 2.4570E-03 2.4317E-03\n', ' 2.4473E-03 2.4043E-03 2.3686E-03 2.3428E-03 2.3553E-03 2.3151E-03 2.2943E-03 2.2631E-03 2.2336E-03 2.2316E-03\n', ' 2.2172E-03 2.1971E-03 2.1744E-03 2.1710E-03 2.1443E-03 2.1000E-03 2.0922E-03 2.0759E-03 2.0387E-03 2.0362E-03\n', ' 1.9895E-03 1.9942E-03 1.9948E-03 1.9549E-03 1.9333E-03 1.9122E-03 1.8853E-03 1.8499E-03 1.8436E-03 1.8509E-03\n', ' 1.8216E-03 1.7942E-03 1.7863E-03 1.7658E-03 1.7584E-03 1.7364E-03 1.7184E-03 1.6866E-03 1.6766E-03 1.6682E-03\n', ' 1.6489E-03 1.6516E-03 1.6458E-03 1.6172E-03 1.5927E-03 1.5700E-03 1.5644E-03 1.5571E-03 1.5555E-03 1.5459E-03\n', ' 1.5212E-03 1.5014E-03 1.4774E-03 1.4688E-03 1.4505E-03 1.4173E-03 1.4149E-03 1.4019E-03 1.3890E-03 1.3674E-03\n', ' 1.3654E-03 1.3564E-03 1.3445E-03 1.3314E-03 1.3347E-03 1.3286E-03 1.3149E-03 1.2993E-03 1.2791E-03 1.2619E-03\n', ' 3.0682E-01 2.3070E-02 1.0015E-02 6.6121E-03 4.7106E-03 4.3606E-03 3.8993E-03 3.6727E-03 3.4450E-03 3.3637E-03\n', ' 3.2508E-03 3.2452E-03 3.2126E-03 3.0648E-03 2.9456E-03 2.9585E-03 2.9646E-03 2.9385E-03 2.9370E-03 2.8502E-03\n', ' 2.8172E-03 2.7795E-03 2.7574E-03 2.7382E-03 2.7056E-03 2.6796E-03 2.6479E-03 2.6971E-03 2.6592E-03 2.6076E-03\n', ' 2.5822E-03 2.5349E-03 2.4951E-03 2.5063E-03 2.5053E-03 2.4324E-03 2.4227E-03 2.4149E-03 2.4058E-03 2.3940E-03\n', ' 2.3859E-03 2.3804E-03 2.3559E-03 2.3242E-03 2.3176E-03 2.2903E-03 2.2557E-03 2.2321E-03 2.2233E-03 2.2019E-03\n', ' 2.1705E-03 2.1483E-03 2.1279E-03 2.1332E-03 2.1202E-03 2.0658E-03 2.0555E-03 2.0506E-03 2.0387E-03 2.0139E-03\n', ' 1.9850E-03 1.9849E-03 1.9527E-03 1.9422E-03 1.9101E-03 1.8871E-03 1.8887E-03 1.8506E-03 1.8300E-03 1.8329E-03\n']

Any assistance on this is greatly appreciated!

Answer 1

This is behaving as expected. From the docs:

readlines: Read until EOF using readline() and return a list containing the lines thus read. If the optional sizehint argument is present, instead of reading up to EOF, whole lines totalling approximately sizehint bytes (possibly after rounding up to an internal buffer size) are read. Objects implementing a file-like interface may choose to ignore sizehint if it cannot be implemented, or cannot be implemented efficiently.

readline: Read one entire line from the file. A trailing newline character is kept in the string (but may be absent when a file ends with an incomplete line).

If you don't want the trailing line, strip on each line will get rid of it.

If you just want to skip past some opening lines, there are a number of ways to do that. Personally I'm partial to itertools.islice:

for line in itertools.islice(infile, 9, None):
    print line

Answer 2

I'm not sure that readlines specifies that newlines will be removed, just that they are used as a delimiter.

you can easily fix this, though:

lines = [line.strip() for line in file.readlines()]

Answer 3

Instead of using readlines, you can use Python's with statement like so:

contents=[]
with open(file_name,'rU') as fin:
    for i,line in enumerate(fin):
        if i>9:
            contents.append(line.rstrip('\n'))

Or, same thing more succinctly:

with open(file_name,'rU') as fin:
    contents=[line.rstrip('\n') for i,line in enumerate(fin) if i>9]

Using with and open on files is a preferred idiom because of the automatic closing and more elegant error collection.

You can use enumerate to keep track of the lines in the file you want to skip or keep.