On my site I use an explicit check to workaround this issue. My code is:
{% if paginator.previous_page %}
{% if paginator.previous_page == 1 %}<a href="/">Newer →</a>
{% else %}<a href="{{ paginator.previous_page_path }}">Newer →</a>
{% endif %}
{% endif %}
We seem to have differences in what is counted as newer.