There is nothing unusual about this at all. At lower scale values, your mesh does not cover a great deal of the screen so it does not produce very many fragments. At larger scales, the entire screen is covered by your mesh and worse still, overdraw becomes a huge factor.
You are fragment bound in this scenario, reducing the complexity of your fragment shader should help and a Z pre-pass to reduce overdraw will also help.