I have asked John Messerly (who is working on Dart's Polymer):
This looks like it's Polymer's fault. Polymer depends on shadow DOM, but there seems to be no good way polyfill that functionality, without wrapping every node in the DOM.
The polyfill should be as fast as it possibly can get, but, due to the wrapping, there is some unavoidable cost. The actual overhead depends on the size of DOM and ratio of DOM operations to other app code. Possibly there are some particularly slow areas that haven't been hit yet, though. Especially event dispatch seems to be tricky to implement.
It would probably be interesting to get more data. (Feel free to reach out to us on the bug-tracker).
You would probably get the optimal performance on a native implementation. Chrome 35 is shipping it. Firefox is experimenting with an implementation.
You could also try to refactor your app so that the shadow DOM/Polymer.dart part is contained in an iFrame. This way it wouldn't affect other things on the page.