Never mix component and controller pagination. Both itself work just fine, but mixing them can cause trouble as you can see.
So either don't include the component and use $paginate alone, or use
$this->Paginator->settings()
inside the actions. You can also directly pass settings in your $components array, as well.