sendto() python

Question

I use python program to do traffic generator in Linux Ubuntu, and the code like below:

import socket, sys

host = sys.argv[1] #Server IP Address
textport = sys.argv[2] #Server Binding Port

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) #socket

try:

    port = int(textport)

except ValueError:

    port = socket.getservbyname(textport, 'udp')

while 1:

   try:
        data = open('auth3.log')#read file

        for each_line in data: #each rows
                try:
                        (role,line_spoken) = each_line.split(': ',1)#split two parts
                        role  =  role.strip()
                        s.sendto(role, (host, port))
                        print('Send: ' + str(role) + "\n" )
                except:
                        pass

    except IOError as err:
        print('file isn\'t exist!!~'+str(err))

    finally:
        if 'data' in locals():   #If data have no object, Don't use data to close!!~
                data.close()

    print "\n"

The size of auth3.log is about 1.8M.

When I send data to the destination server, I use snmp which OID is ''ifInOctets'' to get the traffic information.

But I the traffic recalculate to unit of ''Kbits'' is about 128.

How can I use this program to fill the bandwidth up to 1Gbits?(In other words, I want to fill out the bandwidth)

Thanks for your helping.

Answer 1

This version of your code implements the first two optimizations suggested by Chris Merck.

import socket, sys, itertools

host = sys.argv[1] #Server IP Address
textport = sys.argv[2] #Server Binding Port

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

try:
    port = int(textport)
except ValueError:
    port = socket.getservbyname(textport, 'udp')

# preprocess file data    
with open('auth3.log') as data:
    roles = [role for (role, line_spoken) in line.split(': ', 1) for line in data]

# preprocess everything we can
hp = (host, port)
send = s.sendto

for role in itertools.cycle(roles):
    try:
        send(role, hp)
    except:
        pass

For further optimizations, you might want to process it using Cython, which might further speed up the loop. If the code still doesn't generate enough traffic, you'll probably need to launch several processes in parallel.

Answer 2

Your program is not running fast enough to generate 1Gbps on the wire.

To make it run faster, you can:

1. Remove the call to print after sendto. (Print is slow by nature.)
2. Preprocess your auth3.log file so that you do not need to process it within your inner loop. (Right now you are looping on .split and .strip, both of which are wasting CPU time.
3. Rewrite your program to send larger chunks.

But, I fear the result will still not reach 1Gbps. To really max out your line, try using a traffic generation program such as Colasoft Packet Builder (although I'm not sure even that program will do it. 1Gbps is a lot of traffic.)