Well, as you can see, I'm not an Android User (I know much more about UI on iOS). This is why I was confuse.
After looking a different applications, I found out that Gmail and Google Play do have navigation that looks a little bit like mine (replacing a ViewPager by a new fragment, without creating a new activity).
And for my problem, it wasn't a structure problem. When you create the FragmentStatePagerAdapter, be sure to use getChildFragmentManager() and not getFragmentManager()!
ViewPagerAdapter adapter = new ViewPagerAdapter(getChildFragmentManager());