kswedberg jquery smooth scrolling plugin - highlight links and remove highlight on scroll?

Question

Im using Karl Swedberg's jquery smooth scrolling plugin for same page links: https://github.com/kswedberg/jquery-smooth-scroll

The links have a fixed position so they are always visible. I want the links to be highlighted once you've clicked on them, but for the highlight to be removed when you scroll the page (as you may no longer be on the section the link corresponds to).

Its simple to have the links change color on click. However, removing the color when you scroll is harder as the plugin itself scrolls. Ive tried with this code:

$(document).ready(function() {
$('#scrollcontrolls a').smoothScroll(
    {
    afterScroll: function() { 
        $(window).scroll(function () { 
            $("#scrollcontrolls a").css("color", "black");
        });

    }
});
});

$(document).ready(function() {
$('#scrollcontrolls a').click(function() {
    $(this).css('color','red');
});
});

The problem is that once the window scroll function is initialized, it then fires whenever the smooth scroll plugin scrolls the page. This means after you've clicked on one link, it will then always override the color applied to the link on click, so clicking other links doesn't change their color.

Thanks

EDIT- Ive made a version that sort of works with the code below:

 $(window).bind('scroll', function () { 
$("#scrollcontrolls a").css("color", "black");
 });

 $(document).ready(function() {
$('#scrollcontrolls a').smoothScroll(
    {
    afterScroll: function() { 
        $(window).unbind();
        $(this).css('color','red');
        $(window).bind('scroll', function () { 
            $("#scrollcontrolls a").css("color", "black");
        });
    }
});
 });

However sometimes clicking on the link doesn't change its color, but clicking on it again will. I think the different functions fired afterScroll are not always run sequentially. If this is the problem, how can I make them do so?

Thanks