You can use the following in your fragments:
Actionbar ab = getActivity().getActionBar();
ab.setIcon(icon);
Where icon
is a Drawable or a resource id, when the up indicator is a part of your icon.
You can also use setHomeAsUpIndicator if you use API level > 11:
getActivity().getActionBar().setHomeAsUpIndicator(resId);
If your API level is lower, then I suggest you make use of ActionbarSherlock.