For loop using enumerate through a list with an if statement to search lines for a particular string

Question

I am going to compile a list of a recurring strings (transaction ID). I am flummoxed. I've researched the correct method and feel like this code should work. However, I'm doing something wrong in the second block. This first block correctly compiles a list of the strings that I want.

I cant get this second block to work. If I simplify, I can print each value in the list by using

for idx, val in enumerate(tidarray): print val

It seems like I should now be able to use that value to search each line for that string, then print the line (actually I'll be using it in conjunction with another search term to reduce the number of line reads, but this is my basic test before honing in further.

def main():
    pass

samlfile= "2013-08-18 06:24:27,410 tid:5af193fdc DEBUG org.sourceid.saml20.domain.AttributeMapping] Source attributes:{SAML_AUTHN_CTX=urn:oasis:names:tc:SAML:2.0:ac:classes"
tidarray = []

for line in samlfile:  

    if "tid:" in line:
        str=line
        tid = re.search(r'(tid:.*?)(?= )', str)
        if tid.group() not in tidarray:
            tidarray.append(tid.group())

for line in samlfile:
    for idx, val in enumerate(tidarray):
        if val in line:
            print line

Can someone suggest a correction for the second block of code? I recognize that reading the file twice isn't the most elegant solution... My main goal here is to learn how to enumerate through the list and use each value in the subsequent code.

Answer 1

Iterating over a file twice

Basically what you do is:

for line in somefile: pass # first run
for line in somefile: pass # second run

The first run will complete just fine, the second run will not run at all. This is because the file was read until the end and there's no more data to read lines from. Call somefile.seek(0) to go to the beginning of the file:

for line in somefile: pass # first run
somefile.seek(0)
for line in somefile: pass # second run

Storing things uniquely

Basically, what you seem to want is a way to store the IDs from the file in the a data structure and every id shall only be once in said structure.

If you want to store elements uniquely you use, for example, dictionaries (help(dict)) or sets (help(set)). Example with sets:

myset = set()
myset.add(2)  # set([2])
myset.add(3)  # set([2,3])
myset.add(2)  # set([2,3])