Thanks JRowan, I tried that and didn't work, but helped me realize where the problem was. After trying so many different things, I figured that I need to add this to the OnTouchListener:
if (motionEvent.getAction() == MotionEvent.ACTION_UP) {
view.setVisibility(View.VISIBLE);
Very simple, no more disappearing buttons. Thanks