Ok, completely reworking this answer to try to explain it clearer. A couple of things to know are that transformations 'accumulate' and rotation transforms happen around the origin. So to just explain the affect of accumulating (multiplying) transforms, look at this example:
//draw an ellipse centered at 200,200
g.DrawEllipse(Pens.Red, 195, 195, 10, 10);
//apply translate transform - shifts origin to 200,200
g.TranslateTransform(200, 200);
//draw another ellipse, should draw around first ellipse
//because translate tranforms essentially moves our coordinates 200,200
g.DrawEllipse(Pens.Blue, -7, -7, 14, 14);
//now do rotate transform
g.RotateTransform(90f); //degree to rotate object
//now, anything we draw with coordinates 0,0 is actually going to be draw at 200,200 AND be rotated by 45*
//this line will be vertical, through 200,200, instead of horizontal through 0,0
g.DrawLine(Pens.Green, -20,0,20,0);
//If we add another translate, this time 50x, it would normally translate by 50 in the X direction
//BUT - because we already have transforms applied, including the 90 rotate, it affects this translation
//so this in effect because a 50px translation in Y, because it's rotated 90*
g.TranslateTransform(50, 0);
//so even though we translated 50x, this line will draw 50px below the last line
g.DrawLine(Pens.Green, -20, 0, 20, 0);
So for your case, you want to draw an object Centered at CenterPoint and rotated by Angle. So you would do:
g.TranslateTransform(-CenterPoint.X, -CenterPoint.Y);
g.RotateTransform(Angle);
g.DrawImage(b, -ImageSize/2, -ImageSize/2, ImageSize, ImageSize);
You'd then need to reset the transforms for additional drawing, which you can do with:
g.ResetTransform();
If that doesn't leave the image where you want it, then you'll need to check the values you're using to position it. Are you storing it's center? Or top left? Etc.