Well, I did some work around for this
I changed the style of the actionbar I made to have transparent icon and text
<style name="MyTheme" parent="@style/Theme.AppCompat.Light">
<item name="android:actionBarStyle">@style/MyTheme.ActionBarStyle</item>
</style>
<style name="MyTheme.ActionBarStyle" parent="@android:style/Widget.Holo.Light.ActionBar">
<item name="android:titleTextStyle">@style/MyTheme.ActionBar.TitleTextStyle</item>
<item name="android:icon">@android:color/transparent</item>
<item name="android:background">@drawable/actionbar_bg</item>
</style>
<style name="MyTheme.ActionBar.TitleTextStyle" parent="@android:style/TextAppearance.Holo.Widget.ActionBar.Title">
<item name="android:textColor">@android:color/transparent</item>
</style>
2- Then in setupActionBar() method I set the color back manually
private void setupActionBar() {
getSupportActionBar().setDisplayShowHomeEnabled(false);
this.getSupportActionBar().setDisplayShowTitleEnabled(false);
this.getSupportActionBar().setDisplayShowCustomEnabled(true);
LayoutInflater inflator = (LayoutInflater)this.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
View v = inflator.inflate(R.layout.title_view, null);
TextView tvTitle = (TextView) v.findViewById(R.id.actionbar_title);
tvTitle.setTextColor(getResources().getColor(R.color.white));
....
}