Reducing the complexity of dictionary brute force in python

Question

I have a program as follows which basically compares XORs of all possible words in a standard dictionary and compares the XORed result with that of XOR of Ciphertexts.But i guess the complexity is O(n2). I am not sure how to reduce the complexity.

def find_collision():
    a = int("4ADD55BA941FE954",16) ^ int("5AC643BE8504E35E",16)
    with open("/usr/share/dict/words", "r") as f:
        alist = [line.rstrip() for line in f]
    b = len(alist)

    for i in range(0,b,1):
        for j in range(i,b,1):
        if(((int(alist[i].encode('hex'), 16))^ (int(alist[j].encode('hex'), 16)))==a):
            print("Plain Text1: "+alist[i]+'\n'+"Plain Text2: "+alist[j])
            #print "Yes"
            break

Any help would be much appreciated.

Answer 1

First off, let's try to simplify.

def find_collision():
    key = 0b1000000011011000101100000010000010001000110110000101000001010
    # that's 0x4ADD55BA941FE954^0x5AC643BE8504E35E

Then our handy-dandy itertools module can do the heavy lifting for the big list. This replaces your nested for loops and probably works significantly faster.

from itertools import combinations
##def find_collision()
##    key = 0b1000000011011000101100000010000010001000110110000101000001010
with open("/usr/share/dict/words", "r") as f:
    full_wordlist = combinations( map(str.rstrip,f.readlines()), 2 )
    # Combinations( { ('word1','word2'),('word1','word3'),('word1','word4'),
                    ('word2','word3') ... } )

But we don't really care about the whole thing, do we? All we care about is collisions, so let's do collisions shall we? EDIT: and since there will definitely be words in here we can't turn to hex, do:

#instead of full_wordlist = combinations(...)

import re
with open("usr/share/dict/words","r") as f:
    words = (word for word in map(str.rstrip,f.readlines()) if not re.search(r"[^0-9a-fA-F]",word))
    # you can avoid the need for regex by doing:
    # words = (word for word in map(str.rstrip,f.readlines()) if
    #         not any(char not in "0123456789abcdefABCDEF" for char in word))
    collisions = [keypair for keypair in combinations(words,2)
                 if bin(int(keypair[0],base=16)^int(keypair[1],base=16)) == key]

Then pull out the collisions with something sane, like:

for collision in collisions:
    print("Collision between {0[0]}^{0[1]} and key".format(collision))