.htaccess ERRORDOCUMENT not working

Question

Hope someone can help with this.

I have the following code which checks the user agent of visitors and throws an error

# ANYWHERE IN UA
RewriteCond %{HTTP_USER_AGENT} ^.* (exabot|craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]

# SERVE ERRORDOCUMENT
RewriteRule . - [F,L]

ErrorDocument 500 /errorPage500.php

For some reason unknown to me the error document fails with the following error

Forbidden

You don't have permission to access / on this server.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

Do I need to add more code for this to work?

Thanks in advance

Full code below

RewriteEngine On
#prevent hotlinking
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?website.com [NC]
RewriteRule \.(jpg|jpeg|png|gif)$ - [NC,F,L]

# Redirect from website.com to www.website.com
RewriteCond %{HTTP_HOST} ^ website.com [NC]
RewriteCond %{HTTP_HOST} !^m.website.com [NC]
RewriteCond %{REQUEST_URI} !errorPage
RewriteCond %{HTTP_HOST} !^www [NC]
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1  [R=301,L]


ErrorDocument 400 /errorPage400.php
ErrorDocument 401 /errorPage401.php
ErrorDocument 403 /errorPage403.php
ErrorDocument 404 /errorPage404.php
ErrorDocument 500 /errorPage500.php

php_value session.gc.maxlifetime 604800
php_flag register_globals 0
php_flag session.use_only_cookies 1
php_flag session.use_trans_sid 0


<IfModule mod_deflate.c>
#The following line is enough for .js and .css
AddOutputFilter DEFLATE js css

#The following line also enables compression by file content type, for the following list of Content-Type:s
AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml

#The following lines are to avoid bugs with some browsers
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html 
</IfModule>

<IfModule mod_headers.c>
<FilesMatch "\.(js|css|xml|gz)$">
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>

<filesMatch "\.(jpg|jpeg|png|gif|swf)$">
Header set Cache-Control "max-age=604800, public"
</filesMatch>
# 1 WEEK
<filesMatch "\.(xml|txt|css|js)$">
Header set Cache-Control "max-age=604800, proxy-revalidate"
</filesMatch>
# 1 MIN
<filesMatch "\.(html|htm|php)$">
Header set Cache-Control "max-age=60, private, proxy-revalidate"
</filesMatch>

RewriteBase /

# IF THE UA STARTS WITH THESE
RewriteCond %{HTTP_USER_AGENT} ^(aesop_com_spiderman|alexibot|backweb|bandit|batchftp|bigfoot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(black.?hole|blackwidow|blowfish|botalot|buddy|builtbottough|bullseye) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cheesebot|cherrypicker|chinaclaw|collector|copier|copyrightcheck) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cosmos|crescent|curl|custo|da|diibot|disco|dittospyder|dragonfly) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(drip|easydl|ebingbong|ecatch|eirgrabber|emailcollector|emailsiphon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(emailwolf|erocrawler|exabot|eyenetie|filehound|flashget|flunky) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(frontpage|getright|getweb|go.?zilla|go-ahead-got-it|gotit|grabnet) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(grafula|harvest|hloader|hmview|httplib|httrack|humanlinks|ilsebot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(infonavirobot|infotekies|intelliseek|interget|iria|jennybot|jetcar) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(joc|justview|jyxobot|kenjin|keyword|larbin|leechftp|lexibot|lftp|libweb) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(likse|linkscan|linkwalker|lnspiderguy|lwp|magnet|mag-net|markwatch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mata.?hari|memo|microsoft.?url|midown.?tool|miixpc|mirror|missigua) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mister.?pix|moget|mozilla.?newt|nameprotect|navroad|backdoorbot|nearsite) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(net.?vampire|netants|netcraft|netmechanic|netspider|nextgensearchbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(attach|nicerspro|nimblecrawler|npbot|octopus|offline.?explorer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(offline.?navigator|openfind|outfoxbot|pagegrabber|papa|pavuk) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(pcbrowser|php.?version.?tracker|pockey|propowerbot|prowebwalker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(psbot|pump|queryn|recorder|realdownload|reaper|reget|true_robot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(repomonkey|rma|internetseer|sitesnagger|siphon|slysearch|smartdownload) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(snake|snapbot|snoopy|sogou|spacebison|spankbot|spanner|sqworm|superbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(superhttp|surfbot|asterias|suzuran|szukacz|takeout|teleport) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(telesoft|the.?intraformant|thenomad|tighttwatbot|titan|urldispatcher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(turingos|turnitinbot|urly.?warning|vacuum|vci|voideye|whacker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(libwww-perl|widow|wisenutbot|wwwoffle|xaldon|xenu|zeus|zyborg|anonymouse) [NC,OR]

# STARTS WITH WEB
RewriteCond %{HTTP_USER_AGENT} ^web(zip|emaile|enhancer|fetch|go.?is|auto|bandit|clip|copier|master|reaper|sauger|site.?quester|whack) [NC,OR]

# ANYWHERE IN UA
RewriteCond %{HTTP_USER_AGENT} ^.*(exabot|craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]

# ISSUE 403 / SERVE ERRORDOCUMENT
RewriteRule . - [F,L]

#when access is denied, allow access to just the error page and logo
<Files errorPage.*> 
Order Deny,Allow 
Allow from All 
</Files>
<Files  logo.gif>
Order Deny,Allow 
Allow from All
</Files>

Answer 1

Try changing your code to this:

ErrorDocument 400 /errorPage400.php
ErrorDocument 401 /errorPage401.php
ErrorDocument 403 /errorPage403.php
ErrorDocument 404 /errorPage404.php
ErrorDocument 500 /errorPage500.php

# all of your %{HTTP_USER_AGENT} 
# matches here
# ...
# ANYWHERE IN UA
RewriteCond %{HTTP_USER_AGENT} "exabot|craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures" [NC]
# SERVE ERRORDOCUMENT conditionally
RewriteRule (?!^errorPage403\.php$)^ - [F,L]

Answer 2

Access is forbidden for some agents, so the forbidden message is normal. There is no 403 error handler script and that's why the 500 error is also generated.

If you want 403 error to be handled by a script, you have to add it, because the [F] flag causes the server to return a 403 Forbidden status code.

If that's the case, create the errorPage403.php script and try something like this:

RewriteEngine On
# ANYWHERE IN UA
RewriteCond %{HTTP_USER_AGENT} ^(exabot|craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures) [NC]
# SERVE ERRORDOCUMENT
RewriteCond %{REQUEST_URI}  !/errorPage403.php [NC]
RewriteRule . - [F]

ErrorDocument 500 /errorPage500.php
ErrorDocument 403 /errorPage403.php

Update

According to OP comments to this answer and the question update, the solution is to add a condition before the rule as suggested in my comment and in this answer from the beginning, like this:

# Add next line
RewriteCond %{REQUEST_URI} !/errorPage403.php [NC] 

# To current rule in your file.
RewriteRule . - [F,L].

Answer 3

You've got it throwing the 500 error. Notice the:

ErrorDocument 500 /errorPage500.php