How can I stop nginx logging Amazon Route 53 Health Check requests?

Question

At the moment my AWS health check is hitting my server pretty relentlessly:

...
54.228.16.40 - - [14/Jan/2014:10:17:22 +0000] "GET / HTTP/1.1" 301 178 "-" "Amazon Route 53 Health Check Service"
54.248.220.40 - - [14/Jan/2014:10:17:24 +0000] "GET / HTTP/1.1" 301 178 "-" "Amazon Route 53 Health Check Service"
54.232.40.110 - - [14/Jan/2014:10:17:25 +0000] "GET / HTTP/1.1" 301 178 "-" "Amazon Route 53 Health Check Service"
54.241.32.78 - - [14/Jan/2014:10:17:26 +0000] "GET / HTTP/1.1" 301 178 "-" "Amazon Route 53 Health Check Service"
54.245.168.46 - - [14/Jan/2014:10:17:28 +0000] "GET / HTTP/1.1" 301 178 "-" "Amazon Route 53 Health Check Service"
54.251.31.174 - - [14/Jan/2014:10:17:28 +0000] "GET / HTTP/1.1" 301 178 "-" "Amazon Route 53 Health Check Service"
...

And I'd like to configure NginX to not log any requests with a user agent of "Amazon Route 53 Health Check Service".

My current attempt looks as follows:

# default server for forwarding all requests over to main www domain
server {
    listen 80 default_server;

    server_name _;

    return 301 $scheme://www.example.com$request_uri;
}

# server configured to catch aws health check requests
server {
    listen 80;
    server_name 12.345.67.89;

    location / {
        if ( $http_user_agent ~* 'Amazon Route 53 Health Check Service' ) {
            access_log off;
            return 200 'Service OK';      
        }
    }
}

# actual application server
server {
    listen 80;
    server_name www.example.com;

    location / { 
        ...
    }
}

This looks good to me, and in fact when I CURL the same address that the health check is set up to hit:

curl --user-agent "Amazon Route 53 Health Check Service" http://12.345.67.89:80/

I get what I'd expect:

Service OK

And my request doesn't end up in the logs.

However, my logs continue to be swamped by these requests when they come from the actual AWS health check.

Any ideas on where I'm doing wrong?

Thanks

Answer 1

So it turns out that my health check was set up to hit example.com rather than the ip address: my bad.

For the record, I discovered this by adding the $host variable to my log formats (see end of line):

log_format debug_format '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for" host:"$host"';

access_log /var/log/nginx/access.log debug_format;

Cheers anyway

Answer 2

We can map the user agent variable set by NGINX and set the values to boolean to be used when we define the access log path and format. Check below nginx block for reference.

map $http_user_agent $log_ua {

~Pingdom 0;
~Amazon-Route53 0;
~SomeOtherUA 0;

default 1;

}

server {

...

access_log /var/log/nginx/access.log main combined if=$log_ua;

}