I'm afraid the polyfill/shim will be necessary if you want to support older browsers.
As you point out, it works with latest Firefox and Chrome without prefix. Firefox has also a new policy not to prefix any future technology so that's perhaps a good thing too and there is a proposal to do the same for Chrome.
But if you run older browsers (and a surprisingly large group of users do) the polyfill is necessary.
Here we see IE9 on top followed by IE8, 10 and then older Chrome versions (although this includes the whole last year which explains the Chrome part and also IE9 vs. IE10). Variations may apply to regions so test out the geographic area where you think your application is relevant.