Then I want to go back to the map. Ok, I click the button and it calls an new Intent to go back to the map.
Why ? First, why do you need a button to go back ? Just let the user go back using the device's Back button. Secondly, if you really need to use a back button (which I don't know why would you), why create a new intent to go back ? Just call finish() on the Activity with the button.