You should create a template override for the view, in that you can place the link to edit adding:
&tmpl=component
to the url so that only the component part will be shown; and add the
target="_blank"
to the <a href
or the <form
tag so it goes to a new window.
This will most likely open in a new tab, if you want a new window, build the url and pass it to the window.open
call.