I wouldn't use \"
as a field separator. CLF is constructed well enough that if you separate on whitespace, field 12 is the start of your user agent. If $12 == '""'
, the user agent is blank.
Remember that awk
can accept standard input. So you can have "live" monitoring of your Apache log with:
$ tail -F /path/to/access.log | /path/to/awkscript
Just remember that when invoked this way, an awk script will never reach its END
. But you can process lines as they are added to the log by Apache.
Something like this might help. Add to it as you see fit.
#!/usr/bin/awk -f
BEGIN {
mailcmd="Mail -s \"Security report\" webmaster@example.com";
}
# Detect empty user-agent
$12 == "" {
report="Empty user agent from " $1 "\n";
}
# Detect image hijacking
$7 ~ /\.(png|jpg)$/ && $11 !~ /^http:\/\/www.example.com\// {
report=report "Possible hijacked image from " $1 " (" $11 " -> " $7 ")\n";
}
# Detect too many requests per second from one host
thissecond != $4 {
delete count;
thissecond=$4;
}
{
count[$1]++;
for (ip in count) {
if (count[ip] > 100) {
report=report "Too many requests from " $1 "\n";
delete(count[ip]); # Avoid too many reports
}
}
}
# Send report, if there is one
length(report) {
print report | mailcmd; # Pipe output through a command.
close(mailcmd); # Closing the pipe sends the mail.
report=""; # Blank the report, ready for next.
}
Note that counting requests within a particular second is only marginally helpful; if you have a lot of traffic from China, or university/corporate networks behind firewalls, then many requests might appear to come from a single IP address. And the Mail
command isn't a great way to handle notifications; I include it here only for demonstration purposes. YMMV, salt to taste.