If you don't remove an event handler before you add a new handler you can end up with the same method being called multiple times for the same event. So the answer to whether or not this makes a difference really depends on how often this code is run. As a general rule it isn't a bad practice to remove the existing handlers before adding a new handler.
Take the following example. In this example you would have an issue if this click function was called either directly or indirectly from the button being clicked:
var clickFunction = function () {
// add the event AGAIN: BAD!
$("#test-button").on("click", clickFunction);
};
See this in a JSFiddle: http://jsfiddle.net/4Qv8T/1/