Just create custom layout for your ActionBar menu items and find the menu items to which you want to provide custom layout in onCreateOptionsMenu
and call setActionView(View view)
on them.
@Override
public boolean onCreateOptionsMenu(Menu menu) {
...
MenuItem item = menu.findItem(R.id.YOUR_MENU_ITEM_ID);
item.setActionView(R.layout.YOUR_CUSTOM_LAYOUT);
...
}