As far as I can understand your question, here's a solution for you.
You can first check if there are selected items like
boolean hasCheckedItems = yourListAdapter.getSelectedCount() > 0;
Then you can check for the following conditions
if (hasCheckedItems && ActionModeObject == null)
//there are some selected items, start the ActionMode
else if (!hasCheckedItems && ActionModeObject != null)
// there no selected items, finish the ActionMode
ActionModeObject.finish();
if (ActionModeObject != null)
//Action Button Clicked