Solved the issue by adding a separate Location tag in httpd.conf
<Location /dummy>
SetHandler weblogic-handler
WLExcludePathOrMimeType *.json,*.html,*.png
</Location>
<IfModule mod_weblogic.c>
WebLogicHost <host_name>
WebLogicPort 9001
MatchExpression *.jsp
MatchExpression *.controller
MatchExpression /dummy/*
HungServerRecoverSecs 3600
ConnectTimeoutSecs 25
ConnectRetrySecs 2
Debug OFF
</IfModule>
Interestingly, in the documentation of old version of weblogic plugin, it's actually mentioned that WLExcludePathOrMimeType takes comma separated list (http://docs.oracle.com/cd/E13222_01/wls/docs81/plugins/plugin_params.html#1143055). While in the newer documentation, it has been skipped (http://docs.oracle.com/cd/E13222_01/wls/docs103/plugins/plugin_params.html#wp1143055)