If you don't want to see "today's records" too, then unfortunately this isn't possible with out-of-the-box filters.
However, I suggest a simple workflow: it would wait until the due date and then set a hidden checkbox field called something like "Overdue" to true. Then your filter will be very easy to build. You could also have this workflow send an email reminder to the owner of the record.
Hope that helps!