Magento - Many unique 404 requests crashes php-fpm / server

Question

One of my Magento servers crashed the other day and I had to restart php-fpm before it would recover. Looking into the problem, I noticed a lot of HTTP requests (around 40) for random file names (tar, zip, gzip, dmp etc) which resulted in the standard 404 page getting returned.

I assume this was a script kiddie looking for source code, or database dumps that a developer may have created. No file was found, but the number of unique 404 requests did crash php-fpm.

What is the best preventative action I could take to stop this type of attach taking out the server again?

The fact that each of the URLs was unique means that none of the page were cached and Magento code had to run which is quite slow. At the moment, the 404 page returns the content of the home page.

Is there a better 404 page that I could use that would not create as much of a load on the server?

Answer 1

configure your webserver to block requests to these extensions, neither one is for http access, only sftp/ftp.

apache already has something like this in httpd.conf file:

<FilesMatch "^\.ht">
            Require all denied
    </FilesMatch>

add your extensions:

<FilesMatch "^\.(ht|tar|zip|gz|tmp|sql|swp|sh|svn|git)">
            Require all denied
    </FilesMatch>

nginx setup:

# deny files that only for internal use (work with ssh or use ftp to download)
location ~ ^/\.(svn|git|hg|htpasswd|bash|ssh|php_cs|config) { return 444; }
location ~ ^/.*\.(sh|pl|swp|phar|log|ini|sql|conf|yml|zip|tar|.+gz)$ { return 444; }

configure your firewall to block excessive/multiple connections in some reasonable time frame, or install and configure fail2ban