You need to apply the offset to the body.
$('body').scrollspy({
offset: offsetHeight
});
Also, you will need to subtract at least one from the offsetHeight in the line below, otherwise it will not work when scrolling up.
var scrollPos = $('body > .container').find($(this).attr('href')).offset().top - (offsetHeight - 1);