Your question is good but take an example:
1.Suppose you used routes for your different events for a single view. All your clicks will go into router which is totally not needed.
2.Now you have 10 different views which is having 20 different clicks in it.
Will you assign it in router? Button click or key-press click is not a different route.
They are just event which you should handle in that particular view.
3.Also backbone route is fired only on hash change. If you use routers for events then may be backbone will allow you to click once cause after that route is not changed.So next click event will not get captured.
So its a good practice to keep each functionality separate so that it will serve its best and code will also get manageable.