You can try two things
Set the ImageView to consume touch events.
ImageView.setClickable(true);
When pushing the new fragment disable touch events on the ListView.
ListView.setClickable(false);
If you want to know how to know when the fragment with the ImageView is removed try setTargetFragment. Take a look here: https://stackoverflow.com/a/13733914/935421