It seems i have to get the Xrm object in silverlight directly for it to work in IE9:
window = HtmlPage.Window.Eval("document.parentWindow.opener.Xrm") as ScriptObject;
This doesn't seem to work in IE9, only works in IE8:
window = HtmlPage.Window.Eval("document.parentWindow.opener") as ScriptObject;
xrmObject = window.GetProperty("Xrm") as ScriptObject;