Issues with reducing duplicate output from log file search

Question

This website has been a great help since I'm getting back into programming and I'm attempting to write a simple perl script that will analyze apache log files from a directory (multiple domains), pull the last 1000 lines of each log file, strip the IP addresses from the log file and then compare them with a known block list of bot spammers.

Now so far I've got the script working except for one issue. Lets say I have the IP address 10.128.45.5 in two log files, the script of course analyzes each log file in turn stripping and reducing the IP's to one PER log file but what I'm trying to do is narrow that down even more to one per instance I run this script, regardless if the same IP appears across multiple log files.

Here's the code I've gotten so far, sorry if it's a bit messy.

#!/usr/bin/perl

# Extract IP's from apache access logs for the last hour and matches with forum spam bot list.
# The fun work of Daniel Pearson

use strict;
use warnings;
use Socket;

# Declarations
my ($file,$list,@files,%ips,$match,$path,$sort);
my $timestamp = localtime(time);

# Check to see if matching file exists
$list ='list';

if (-e $list) {
Delete the file so we can download a new one if it exists
print "File Exists!";
print "Deleting File $list\n";
unlink($list);
}
sleep(5);

system ("wget http://www.domain.com/list");
sleep(5);

my $dir = $ARGV[0] or die "Need to specify the log file directory\n";

opendir(DIR, "$dir");
@files = grep(/\.*$/,readdir(DIR));
closedir(DIR);

foreach my $file(@files) {
my $sum = 0;
if (-d $file) {
print "Skipping Directory $file\n";
}
else {

$path = "$dir$file";
open my $path, "-|", "/usr/bin/tail", "-1000", "$path" or die "could not start tail on $path: $!";

my %ips;


while (my $line = <$path>) {
chomp $line;
if ($line =~  m/(?!0+\.0+\.0+\.0+$)(([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5]))/g) {

my $ip = $1;

$ips{$ip} = $ip;
        }
}
}

foreach my $key (sort keys %ips) {
open ("files","$list");
while (my $sort = <files>) {
chomp $sort;
if ($key =~ $sort) {
open my $fh, '>>', 'banned.out';
print "Match Found we need to block it $key\n";
print $fh "$key:$timestamp\n";
close $fh;
        }
    }
}
}

Any advice that could be given I would be grateful for.

Answer 1

To achieve the task:

• Move my %ips outside of (above) the foreach my $file (@files) loop.

• Move foreach my $key ( sort keys %ips ) outside of (below) the foreach my $file (@files) loop.