I would recommend you to use imagettftext() if you wish to write text to your image using a truetype font.
imagettftext (
resource $im ,
int $size ,
int $angle ,
int $x ,
int $y ,
int $col ,
string $fontfile ,
string $text
)
This allows you to set the font size specifically. Should do for your purposes.
Note
Depending on your version of the GD library, you need to specify the font size in either pixels or points.
From the Docs:
The font size. Depending on your version of GD, this should be specified as the pixel size (GD1) or point size (GD2).