You are using anchor links in the URL (#) and therefore every time you access a page with an anchor link, it will redirect you to the section or slide it corresponds. This is the purpose of anchor links in any site apart from adding extra info to the user.
You could only do what you want by using the plugin without anchor links. (So the URL will always be the same no matter in which section or slide you are in)