There are a few 3 main options for doing security trimming of SharePoint search results. They include:
- ISecurityTrimmerPre - New in SharePoint 2013. Allows adding additional claims for the user before a search query is executed
- ISecurityTrimmerPost - Also known as ISecurityTrimmer2 in earlier versions of SharePoint. Allows trimming search results on a 1 by 1 basis after a search query is executed
- Business Connectivity Services Indexing Connectors - Allows you to supply ACLs to the search indexer when it is crawling the content source
There is a great set of blog articles that show how to implement each option.
Also, here's a great video of a SharePoint Conference 2012 session on custom security trimming.
HTH
-Eugene