Use another variable to determine whether you should currently block the arrow keys:
var useArrowKeysToScroll = false;
/* ... */
function(e){
keys[e.keyCode] = true;
if(!useArrowKeysToScroll &&
((e.keyCode >= 37 && e.keyCode <= 40) ||
e.keyCode == 32)){
e.preventDefault();
}
/* handle other keys etc... */
}, /* ... */
If useArrowKeysToScroll
is false it the arrow keys and space bars won't result in scrolling.