I'm guessing (from your tags) that you're programming a calculator of some sort and that you want to achieve right-aligned text, not right-to-left-oriented text. Right-to-left orientation is used for e.g. arabic languages, which are written (you guessed it) from right to left, instead of the "western" way of writing from left to right.
I suggest you remove the applyComponentOrientation()
and look at setHorizontalAlignment instead.
PS: that being said, I can't really tell why '5'+'.' is '.5', but '5'+'.'+'2' is displayed as '5.2'.