I noticed that "<script>" tags do not get processed by crawler4j. This was where all of the ".js" files occurred. So I don't think the problem is only limited to ".js" files - I think it's anything within the "<script>" tags (which usually happens to be ".js" files).
It does initially look like modifying HtmlContentHandler's Enumeration and startElement() method will solve problem. I tried that and it did not work. While debugging it, I observed that either the Tika Parser or TagSoup (which Tika uses) is not picking up the script tags. As a result it never even reaches crawler4j to get processed.
As a workaround, I used JSoup to parse the HTML for all "<script>" tags in my visit() method and then I schedule a crawl on those files.
I think the real solution is identifying why Tika (or TagSoup) is not picking up the script tags. It could be the way in which it is getting called by crawler4j. Once that is resolved, then modifying the HtmlContentHandler will work.