Issue with "if" statement in Python

Question

I have one file and I am trying to populate second file using its data in Python. While populating I am applying some if statements to manipulate one column in the 2nd file.

What I want to perform using these if statements: Check 31st column in file and if its value is 0 then return 0 value in 32nd column, if its value is blank then return 2 value in 32nd column and for every other non-zero and non-blank value return 1 value in 32nd column.

with open('DE_Combined_' + time.strftime("%Y-%m-%d")+".csv", "rb") as in_file, open('DE_Combined_PD_' + time.strftime("%Y-%m-%d")+".csv", "wb") as out_file:
   reader = csv.reader(in_file)
   writer = csv.writer(out_file)
   headers = next(reader, None)  # returns the headers or `None` if the input is empty
   if headers:
        writer.writerow(headers)
   for row in reader:
       if row[30] != 0:
           row[31] = 1
       else:
           row[31] = 0
       if row[30] == "":
           row[31] = 2
       writer.writerow(row)

This seems like a pretty straightforward problem but my output file is giving me the wrong results. It has 1 value in 32nd column for 0 value in 31st column. It should have been 0 in both columns. Although 2 value is coming against the blank value in 31st column.

Please help me in resolving this problem as I am clueless now after several attempts to find something wrong with the logic of if statements I have used.

Snippet of actual O/P: ( last 2 entries in both the rows are the columns concerned)

05/23/2013  May 2013    2013    4   2   Thursday    UK  O2  £   NOKIA   100 NOKIA 100   Smartphone  Symbian NA          9.99            N       Pay & Go    Pay & Go                    0.64222 0   1
05/23/2013  May 2013    2013    4   2   Thursday    UK  O2  £   NOKIA   100 NOKIA 100   Smartphone  Symbian NA  Pink        9.99            N       Pay & Go    Pay & Go                    0.64222 0   1

I/P Snippet:(First 2 rows) ( Last 2 columns are the concerned columns)

Date,Month,Fiscal_Year,Calendar_Year,FY_Quarter,CY_Quarter,Day_of_Week,Geography,MO,Currency,Device_OEM,Device_Name,GDN,Device_Type,Device_OS,Device_Franchise,Device_Color,Device_Storage_in_GB,Device_Price,Device_Monthly_Price,Additional,Device_Refurb,Plan_COMPARISON,Plan_Name,Plan_Contract_Length,Plan_Monthly_Price,Plan_Data_in_GB,Plan_Minutes,Plan_Texts,Exchange_Rate_vs_1USD,Difference,Difference_Flag
05/23/2013,May,2013,2013,4,2,Thursday,UK,O2,£,NOKIA,100,NOKIA 100,Smartphone,Symbian,NA,,,9.99,,,N,,Pay & Go,Pay & Go,,,,,0.64222,0.0,
05/23/2013,May,2013,2013,4,2,Thursday,UK,O2,£,NOKIA,100,NOKIA 100,Smartphone,Symbian,NA,Pink,,9.99,,,N,,Pay & Go,Pay & Go,,,,,0.64222,0.0,

Answer 1

If I change your logic to this (remember, everything is strings unless you transform them using int(), float() etc.)

for row in reader:
    print "'%s'" % row[30]
    if float(row[30]) != 0:
        row[31] = '1'
    else:
        row[31] = '0'
    if row[30] == "":
        row[31] = 2
    writer.writerow(row)

I get this output:

05/23/2013,May,2013,2013,4,2,Thursday,UK,O2,£,NOKIA,100,NOKIA 100,Smartphone,Symbian,NA,,,9.99,,,N,,Pay & Go,Pay & Go,,,,,0.64222,0.0,0
05/23/2013,May,2013,2013,4,2,Thursday,UK,O2,£,NOKIA,100,NOKIA 100,Smartphone,Symbian,NA,Pink,,9.99,,,N,,Pay & Go,Pay & Go,,,,,0.64222,0.0,0

where the last elements are 0; that was the problem, right?

Answer 2

From the docs:

Each row read from the csv file is returned as a list of strings. No automatic data type conversion is performed.

Perhaps your if row[30] != 0 should be changed to if row[30] != "0"

Answer 3

Separate your tasks into functions that read, manipulate, then write the data.

To read your table into a list:

def importCSV(fname):
    """ Returns a tuple using a CSV as input. First tuple is the
    header. 
    Second tuple is a dictionary with all the data.
    """
    data = []
    with open(fname, 'rb') as csvfile:
        reader = csv.DictReader(csvfile, delimiter=',')
        header = reader.fieldnames
        for lines in reader:
            data.append(lines)
        return (header, data)

header, data = importCSV('myfile.csv')

now manipulate your data:

for line in data:
    do something to line...
....

then write your data:

def CreateCSVFromList(header, source, outputfile):
    """Creates a CSV from a 2D List.
    Header should be a simple list for each column in the resulting list.
    Source is the list.
    Output file is what is written. Include the .csv extension please.
    No error-checking is being done at the moment.
    """
    import csv
    with open(outputfile, 'wb') as csvfile:
        writer = csv.writer(csvfile, dialect='excel')
        writer.writerow(header)
        for row in source:
            writer.writerow(row)


CreateCSVFromList(header, data, 'output.csv')

Answer 4

Try this:

_buffer = ''
for row in reader:
    if reader.line_num == 31:
        if row == ['']:             #check whether 31st row is empty
            _buffer = 2               #if that's true write `2`to line 32

        elif row == ['0']:      #if that's false check wheter 31st row = 0
            _buffer = 1           #if true, write `1`to line 32

        else:                 
            _buffer = 1 

    if reader.line_num == 32:
        row = _buffer

    writer.writerow(row)