Ilan Frumer suggestion is probably a good one.
Now, about your specific question, your patterns certainly need to be adapted as follow:
Example:
'!**/ie/*'
will ignore any file in a folder named "ie" anywhere in your subdirectories (which is likely what you want, is this correct?)
Same goes for:
'!**/iecompat.js'
that will ignore a file named iecompat.js anywhere in folders / subfolders.
You should start here to better understand file selection mechanisms in grunt.