First of all you need to store the caret position so that even when the input doesn't have focus you know it's last position. You can do this by listening to a range of events that may mean the caret position has changed.
Then when the user clicks the button to add a special character, you can place the special character at the caret position you stored. You can also return focus, setting the caret back to it's last position.
I've found a great example of how to keep track of the caret position: Cursor Position Control Example
Hopefully this will get you going in the right direction.