You have to secure your elmah page with login / password (because it contains session cookie, form parameters ... ).
I did like this : create a folder on your website root named "log" change its security setting for allowing only domain admin.
then in you web.config change your httphandler from "elmah.axd" to "log/elmah.axd" so when you'll try to access elmh it'll ask for your credentials.