If you want to 'append' the button right after the select tag, this is what you should try to do.
$('<button>Delete row</button>').appendTo( $(new_row.find('td:first')) );
Add this piece of code right before
tbody.append(new_row);
and you should be able to see the button right bellow the select tag. The logic is quite simple - You're telling jQuery to find the very first cell of the newly created row, and append a button right at the end of it. And since we have a select box there, the button will appear right after it. The code is very simple, so it should be a good start for you.