Turns out this was one of those situations where the shader was behaving just as I expected it to but I misinterpreted exactly what that meant.
In this case I was using the same shader to render both the initial background texture and the overlaying text texture. Because the alpha channel on the background texture is a solid 1.0 (white) all the way across it was rendering the dirt texture as a solid red. Then on the second pass it would render the text properly, but I couldn't see it because it was also being rendered as red.
I fixed my problem in this situation by instead drawing the letter in the alpha channel of the letter texture with black (i.e. Transparent) and then in the shader doing alpha testing and inverting where necessary. The key with this method is to Enable blending before the letter is rendered, and to disable it after.
In the end the shader works something like this.
If fragment alpha < 1
Then, Color = vec4(in_color,(tex_color.a = (tex.color.a) ? (1.0 - tex_color.a) : 1.0));Else, (Alpha = 1) invert the alpha channel
tex_color.a = 0;
Color = tex_color;
This makes it so that whatever is drawn in black in the alpha channel while blending is enabled will come out colored instead of transparent.
In the end, multitexturing is probably the way to go, but in my case I wanted to try to do it with only 1 texture enabled at a time.