There is nothing obviously wrong with your code. A common pitfall is the wireframe color. If it is left to be semitransparent (default), the necessary sorting would slow the rendering down. But you have already set it to Visible = false
.
So on my machine (win 7, T430 notebook, i7 and similar graphics) it takes <2 sec to update (Release with no debugger attached!). I am afraid, that's just what it takes. There is a lot of stuff going on in the back ...
@Edit It might be faster to precompute the colors and provide them as discrete color using ILSurface.UpdateRGBA(). You will have to try and use a profiler to investigate the bottleneck. Another option - since you are after a simple imagesc-style plot - is to build the imagesc on your own: ILTriangles(-strip) ist much more slim and probably gives more option to increase the update speed. However, you will have to do a considerable amount of reordering / vertex generation / color computation on your own. Also, this won't give you the colorbar support of ILSurface.
@Edit: You can use the ILImageSCPlot class as a slim replacement for ILSurface. The documentation is here: http://ilnumerics.net/imagesc-plots.html