You can get it to misbehave a lot worse than that:
This goes wrong because of your code using e.ClipRectangle. Note that it appears twice in your snippet. That variable does not give you the border rectangle. It tells you how much of your client area needs to be re-drawn. It is an optimization opportunity, you can draw less by omitting the parts of the client area that don't need to be refreshed.
It is usually the same size as the display rectangle, which is why it looked like it worked just fine. But not when you put it inside a scrollable container, Windows optimizes scrolls by blitting the parts of the client area that simply can be moved. And then generates a paint for the parts that are revealed by the scroll. With a small e.ClipRectangle. You can see that in the screenshot, note the small rectangles.
Replace e.ClipRectangle with Me.DisplayRectangle.