Try following:
$(elem).on("touchend", function() {
... do your stuff ...
document.activeElement.blur();
});
You could also try setting pointer-events: none;
with CSS to elements that are behind the menu. And then set it back to "auto", when drawer is closed.