Apparently the animation was causing this weird behavior. Not sure why but until I figure it out I had to remove the setting of this animation.
galleryViewPager.setClickable(true);
//galleryViewPager.setAnimation(new DepthPageTransformer());
galleryViewPager.setOnClickListener(new OnClickListener() {...
Now the fling animation is using the standard. Must be the way the Motorola Razr implements the adapter fling animation that messes up the listener order.