I would argue this is a more safe way of doing it
Setting variable to false
var dragging = false;
Setting var to true ontouchmove (allows you to reuse this code everywhere in your app)
$("body").on("touchmove", function(){
dragging = true;
});
Your button
$("#button").on("touchend", function(){
if (dragging)
return;
// your button action code
});
Resetting variable (important)
$("body").on("touchstart", function(){
dragging = false;
});