How to format an output in Python?

Question

I am having difficulty in formatting some code in Python: My code is here:

keys = ['(Lag)=(\d+\.?\d*)','\t','(Autocorrelation Index): (\d+\.?\d*)',       '(Autocorrelation Index): (\d+\.?\d*)',     '(Semivariance): (\d+\.?\d*)']

import re
string1 = ''.join(open("dummy.txt").readlines())
found = []
for key in keys:
found.extend(re.findall(key, string1))
for result in found:
    print '%s  =  %s' % (result[0],result[1])
raw_input()

So far, I am getting this output:

Lag = 1

Lag = 2

Lag = 3

Autocorrelation Index = #value

......

......

Semivariance = #value

But the desired output I want is:

 Lag        AutoCorrelation Index   AutoCorrelation Index   Semivariance
  1              #value                   #value               #value
  2              #value                   #value               #value
  3              #value                   #value               #value

If this output can be possible in a CSV file or a txt file, that would be great!

I think this is a way how you should output the loops, but I am not that great with loops.

My updated code (OLD version)

based on @mutzmatron answer

keys = ['(Lag)=(\d+\.?\d*)',
    '(Autocorrelation Index): (\d+\.?\d*)',
    '(Semivariance): (\d+\.?\d*)']

import re
string1 = open("dummy.txt").readlines().join()
found = []
for key in keys:
    found.extend(re.findall(key, string1))
raw_input()
for result in found:
    print '%s  =  %s' % (result[0], result[1])

raw_input()

not yet compiling! I am using IDLE python 2.6 , don't know the error messages since I don't know the pause command in the prompt!

Original Question

I am totally new to python and have a question. I am trying to process a large text file. Here is just a snippet of it:

Band: WDRVI20((0.2*b4-b3)/((0.2*b4)+b3))
Basic Statistics:
  Min: -0.963805
  Max: 0.658219
  Mean: 0.094306
  Standard Deviation: 0.131797
Spatial Statistics, ***Lag=1***:
  Total Number of Observations (Pixels): 769995
  Number of Neighboring Pairs: 1538146
  Moran's I:
    ***Autocorrelation Index: 0.8482564597***
    Expected Value, if band is uncorrelated: -0.000001
    Standard Deviation of Expected Value (Normalized): 0.000806
    Standard Deviation of Expected Value (Randomized): 0.000806
    Z Significance Test (Normalized): 1052.029088
    Z Significance Test (Randomized): 1052.034915
  Geary's C:
    ***Autocorrelation Index: 0.1517324729***
    Expected Value, if band is uncorrelated: 1.000000
    Standard Deviation of Expected Value (Normalized): 0.000807
    Standard Deviation of Expected Value (Randomized): 0.000809
    Z Significance Test (Normalized): 1051.414163
    Z Significance Test (Randomized): 1048.752451
  ***Semivariance: 0.0026356529***
Spatial Statistics, Lag=2:
  Total Number of Observations (Pixels): 769995
  Number of Neighboring Pairs: 3068924
  Moran's I:
 Autocorrelation Index: 0.6230691635
   Expected Value, if band is uncorrelated: -0.000001
   Standard Deviation of Expected Value (Normalized): 0.000571
   Standard Deviation of Expected Value (Randomized): 0.000571
 Z Significance Test (Normalized): 1091.521976
 Z Significance Test (Randomized): 1091.528022
  Geary's C:
Autocorrelation Index: 0.3769372504
  Expected Value, if band is uncorrelated: 1.000000
  Standard Deviation of Expected Value (Normalized): 0.000574
  Standard Deviation of Expected Value (Randomized): 0.000587
 Z Significance Test (Normalized): 1085.700399
 Z Significance Test (Randomized): 1061.931158
Semivariance: 0.0065475488

I need to extract the information in between the star *** values ( eg : Autocorrelation Index, Semivariance values ) and process it , maybe write it to a different text file or excel file. Can I do that? Help would be really appreciated.

Answer 1

In order to format the data by section perhaps it's easiest to work on the segments as follows

keys =['(Lag)=(\d+\.?\d*)',
    '(Autocorrelation Index): (\d+\.?\d*)',
    '(Semivariance): (\d+\.?\d*)']

import re
string1 = ''.join(open("dummy.txt").readlines())

sections = string1.split('Spatial Statistics')

output = []
heads = []

for isec, sec in enumerate(sections):
    found = []
    output.append([])
    for key in keys:
        found.extend(re.findall(key, sec))
    for result in found:
        print '%s  =  %s' % (result[0],result[1])
        output[-1].append(result[1])
    if len(found) > 0 & len(heads) == 0:
        heads = [result[0] for result in found]    

fout = open('output.csv', 'w')
wrt = csv.writer(fout)
wrt.writerow(heads)
wrt.writerows(outputs)
fout.close()

Answer 2

Populate a list of keys (regular expressions) you want to find. For example,

keys = ['(Lag)=(\d+\.?\d*)',
        '(Autocorrelation Index): (\d+\.?\d*)',
        '(Semivariance): (\d+\.?\d*)']

And then search for these using a regular expression,

import re
string1 = ''.join(open(FILE).readlines())
found = []
for key in keys:
    found.extend(re.findall(key, string1))

for result in found:
    print '%s  =  %s' % (result[0], result[1])

You should then have a list of the entries you want, with which you can do what you need to next!

Result:

Lag  =  1
Autocorrelation Index  =  0.8482564597
Autocorrelation Index  =  0.1517324729
Semivariance  =  0.0026356529

CSV

To output to CSV, use the csv module;

import csv
outfile = open('fileout.csv', 'w')
wrt = csv.writer(outfile)
wrt.writerows(found)
outfile.close()