Unless you manually clear your screen surface, flip will not change its content.
Thus, if you neglect to draw to a certain location, it will remain the same.
If you want to get rid of this effect, usually called "hall of mirrors", you will have to keep track of what portions of the screen have not been drawn to yet and draw over these yourself.
It may be easier to define background sprites around your map's contours and block your camera from going off too far.
Since you use a "dirty/clean" approach to only redrawing what's changed, you won't have the option to just fill the whole screen surface before you draw your frame, because that would draw over anything that's stayed the same since the last frame.