The shown code is incomplete and makes it impossible to track. But I'm assuming that the NaN is being passed to Floor() to begin with. Pass a valid number/numeric string to floor() to make it work. Error is in the logic of code not shown in the question.
EDIT:
After looking at your code it's clear that the problem is due to a misplaced assignment operation. If the 'if' part is called, everything is fine, but if the 'else' part is called, 'num' is never created or assigned a value, causing it to be NaN. To fix this issue I'd recommend doing this:
Change this:
if(document.getElementById("div").style.top === "40px"){
var num = Math.floor(document.getElementById("num").innerHTML);
document.getElementById("num").innerHTML = num+1;
}
else{
document.getElementById("num").innerHTML = num-1;
}
To this:
var num = Math.floor(document.getElementById("num").innerHTML);
if(document.getElementById("div").style.top === "40px"){
document.getElementById("num").innerHTML = num+1;
}
else{
document.getElementById("num").innerHTML = num-1;
}