You need to set a background color before drawing your text:
drawing.Clear(Color.White);
drawing.DrawString(text, barcodeFont, textBrush, 0, 0);
Or if you want a transparent background you need to turn off font smoothing.
drawing.TextRenderingHint = System.Drawing.Text.TextRenderingHint.SingleBitPerPixel;
drawing.DrawString(text, barcodeFont, textBrush, 0, 0);