You can use the labels
plotting style for this:
plot '-' matrix with image, 'numbers.txt' u 1:2:3 with labels
After the with labels
you can use any formatting specifier which you can also use for labels:
plot '-' matrix with image, 'numbers.txt' u 1:2:3 with labels textcolor rgb 'red' font ',16'
To remove the legend use unset key
.