When you click on your delete button, your code is creating a bind that makes any click on any widget, run the code to remove it.
$(document).on( "click", ".gridster ul li", function() {
$(this).addClass("activ");
gridster.remove_widget($('.activ'));
});
You can try change your html to:
<li data-row="1" data-col="4" data-sizex="1" data-sizey="1" class="gs_w"><button class="delete-button" style="float: right;">+</button><h3>4</h3><span class="gs-resize-handle gs-resize-handle-both"></span></li>
<li data-row="1" data-col="6" data-sizex="1" data-sizey="1" class="gs_w"><button class="add-button" style="float: right;">-</button><h3>6</h3></li>
and, in your javascript code, after initializing your gridster add:
$(document).on( "click", ".gridster .delete-button", function() {
var gridster = $(".gridster ul").gridster().data('gridster');
gridster.remove_widget($(this).parent());
});
$(document).on("click", ".gridster .add-button", function() {
var gridster = $(".gridster ul").gridster().data('gridster');
gridster.add_widget('<li class="gs_w">The HTML of the widget...</li>', 2, 1);
});
Hope it helps.