Python: Strange output for threaded ARP ping

Question

thanks for taking the time to look at this!

The issue I'm having is with the output of my threaded ARP script. What I'm trying to achieve is to have the IP address, MAC address and NIC vendor of each alive host output to command prompt.

I have an older ARP script without threading, which takes around 90 seconds and prints my ideal output.

Below is my newer script, based off of the aforementioned script, with threading. Unfortunately, I have no idea why there are no values being shown in the output. If anybody can help I will be extremely grateful!

Thanks in advance!

def arp2(ip):

    # An ARP scanner for the network.
    ips = []

    global ans, unans
    ans, unans = srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=ip), timeout=2, verbose=0)

    for snd, rcv in ans:
    #Assign MAC address and IP address to variables mac and ipaddr

        mac = rcv.sprintf(r"%Ether.src%")
        ipaddr = rcv.sprintf(r"%ARP.psrc%")

        #Get NIC vendor code from MAC address
        niccode = mac[:8]
        niccode = niccode.upper()

        print ips
        ips.append("end")

        #ARPips file amendments
        with open( 'C:\Python26\ARPips.prn', 'w+') as f:
            f.write("\n".join(map(lambda x: str(x), ips)) + "\n")

        #String lookup for NIC vendors. DO NOT CHANGE 'r' TO ANY OTHER VALUE.
        with open('C:\Users\TomVB\Desktop\OID2.prn', 'r') as file:
            for line in file:
                if niccode in file:
                    return mac, ipaddr, line[8:]




def main():

    print "Discovering..."
    print ""
    print "MAC Address \t \t  IP Address \t  NIC Vendor"


    pool = Pool(processes=12)

    Subnetlist = []

    for i in range(255):
        Subnetlist.append(str(IPInt+str(i)))

    global ARPresults
    ARPresults = pool.map(arp2, Subnetlist)

    pool.close()
    pool.join()


    print "\n".join(ARPresults)

if __name__ == '__main__':
    main()

This script gives me the following output:

Mac Address    IP address      NIC Vendor

[][]

[]

[]

[]
 []
[][]
[]

[]  

and so on like this for around 200 lines.

Answer 1

First off, it looks like your you're using multi-processing and not threading. Those two behave quite differently and I suggest you look into that. Anyway, for the problem at hand, the cause is elsewhere.

The arp2 method is executed in parallel. I see two problems in that method:

print "%s \t  %s  \t  %s" % (mac, ipaddr, line[8:])

This statement prints to standard-output. In our code it might be executed by up to 12 processes simultaneously. Python gives you no gurantees that the print statement is atomic. It might very well happen that one process has written half the line when the next process writes his line. In short, you get a mess in the output.

The same holds for

with open( 'C:\Python26\ARPips.prn', 'w+') as f:
    f.write("\n".join(map(lambda x: str(x), ips)) + "\n")

Again there are no guarantees that the processes won't step on each others toes. The file content might get scambled.

The easiest solution is not to do any file or console output inside the arp2 method. Instead return the results. pool.map will collect those results for you in a safe manner. It behaves like the regular map function. Then you can output them to files and console.

If you want output while the scan is running you have to synchronize the processes(for example with multiprocessing.Lock. So that only one process is ever writing/printing at the same time.

Also:

• Put an 'r' in front of string literals with windows-style paths: x = r'C:\Users\TomVB\Desktop\OID2.prn'. Backslashes are used for escaping in Python.

• Load the content of C:\Users\TomVB\Desktop\OID2.prn into a dict. It will be much faster.

• map(lambda x: str(x), ips) is equal to map(str, ips)