Pretty sure you're on the right track. Instead of this
'customCode' => '{literal}<input id="Button" title="Button" class="button" type="button" name="Button" value="Some Button" onClick="window.open("http://google.com","","width=800,height=600");">{/literal}',
Try this
'customCode' => '{literal}<input id="Button" title="Button" class="button" type="button" name="Button" value="Some Button" onClick="window.open(\'http://google.com\',\'\',\'width=800,height=600\');">{/literal}',
The difference being the mixed quotes... you had double quotes embedded in double quotes. Instead, use escaped single quotes inside of them.