Import log file data into an application

Question 1

Not sure what your issue is, but here's some pointers on your Switch statement and other code.

As far as I know, the use Switch feature/module has been replaced in perl v5.10.1 with the given/when feature, and is now deprecated. Personally, I have always found these statements a little odd and unreliable. And not irreplaceable by any means, as perl is such a flexible language.

In your case, I would recommend a hash lookup instead:

my %camp = ("ssss_uk_01B" => 1,
            "ssss_uk_01C" => 2,
            ...);
my %cand = ("Brown"       => 1,
            "Cameron"     => 2,
            ...);

$campaign_id  = $camp{$campaign};
$candidate_id = $cand{$choice}

If you need to provide default values, you can use the defined-or assignment operator:

$campaign_id //= "default value here";

Your substr assignments are a bit wrong. First off, you use all four arguments, which in perldoc is described like this:

substr EXPR,OFFSET,LENGTH,REPLACEMENT

Which means that you replace your match with '' the empty string. This affects the variable itself, and would delete your wanted data, if left on its own. Length in your case is not needed either, since it is the end of the string you are after. Lastly, you are saved by the fact that you use the return value of the substr statement.

Finally, when I try out your assignments, I get a one-off error on your first assignment ($campaign). The output is :ssss_uk_01, and you expect ssss_uk_01.

To use the substr function correctly, you should use:

$campaign = substr $campaign, 9;
$validity = substr $validity, 9;
$choice   = substr $choice, 7;

Question 2

I agree with TLP's suggestion on using hashes for lookup tables. But in a real application those lookup tables should be populated with a database lookup to avoid needing to alter the application when the data in the other tables change.

I'd also tend to be a little less trusting of the format of the data in the log. Using split alleviates the need to know the exact length of the label+colon size. Along that line I also suggest normalizing the data before using it as a key in the lookup hash. In this example I just made it all lower case.

While you're using warnings, you might as well use the warn function to bark out about error conditions. In a real application you might tap warnings into an email alert system so that you can always know when your unattended software is having issues.

The or+die idiom is an excellent practice when doing your file operations. But it will be more informative when these dies occur if you include $OS_ERROR (a.k.a. $!), even for print. In that way you will know the difference between permission denied, doesn't exist, or disk full, etc.

Those are suggestions that speak to issues of writing maintainable and resilient programs. But I personally believe that aesthetics of the code also contribute to the maintainability. Meaningful naming conventions and code formatting can make a big difference when you open up this program a year or two from now to make an improvement or a bug fix. I prefer to create code that lacks unnecessary clutter such as comments that state the obvious or the use of commenting to remove blocks of code.

Keep up the good work. The world needs more software automation.

#!/usr/bin/perl

use strict;
use warnings;
use DBI;

#
# _voting.pl
#
my $logfile = '/var/www/voting/votes.txt';

my $errors = 0;

my %campaign_id_for = (
    'ssss_uk_01b' => 1,
    'ssss_uk_01c' => 2,
    'ssss_uk_01d' => 3,
    'ssss_uk_01e' => 4,
    'ssss_uk_01f' => 5,
    'ssss_uk_01g' => 6,
);

my %candidate_id_for = (
    'brown'   => 1,
    'cameron' => 2,
    'balls'   => 3,
    'green'   => 4,
    'boring'  => 5,
    'tupele'  => 6,
);

my $dbh = DBI->connect( 
    'DBI:mysql:database=sms_voting;host=localhost',
    'sisisi', 
    '*********', 
    { 'RaiseError' => 1 } 
);

my $sth = $dbh->prepare(q{
    INSERT INTO voting (
        epoch,
        validity,
        choice,
        campaigns_id,
        candidates_id
    ) VALUES (
        ?,
        ?,
        ?,
        ?,
        ?
    )
});

my $fh;

open $fh, '<', $logfile
    or die "open $logfile: $!";

LINE:
for my $line (<$fh>) {

    my ($vote, $epoch,  $campaign, $validity, $choice,
        $CONN, $MSISDN, $GUID,     $Shortcode
    ) = split /\s+/, $line;

    ($campaign) = reverse split /:/, $campaign;
    ($validity) = reverse split /:/, $validity;
    ($choice)   = reverse split /:/, $choice;

    my $campaign_id  = $campaign_id_for{ lc $campaign };
    my $candidate_id = $candidate_id_for{ lc $choice };

    if ( $epoch && $validity && $choice && $campaign_id && $candidate_id ) {

        $sth->execute(
            $epoch,
            $validity,
            $choice,
            $campaign_id,
            $candidate_id
        );

        print "$epoch $validity $choice\n"
            or die "print: $!";

        next LINE;
    }
    else {

        warn "failed to parse: $line\n";
        $errors++;
    }
}

close $fh
    or die "close $logfile: $!";

# debug
print "error count: $errors\n"
    or die "print: $!";