After sorting out the other issues, the problem was using offset() instead of position() when adding to the scrollTop() of the container.
This now works correctly:
function GetOffset($element, $parent) {
return {
top: $element.position().top + $parent.scrollTop(),
left: $element.position().left + $parent.scrollLeft()
};
}