How to convert UTF-8 string to PChar in Delphi 2009

Question

I receive a string, which is displayed as '{'#0'S'#0'a'#0'm'#0'p'#0'l'#0'e'#0'-'#0'M'#0'e'#0's'#0's'#0'a'#0'g'#0'e'#0'}'#0 in the debugger.

I need to print it out in the debug output (OutputDebugString).

When I run OutputDebugString(PChar(mymsg)), only the first character of the received string is displayed (probably because of the #0 end-of-string marker).

How can I convert that string into something OutputDebugString can work with?

Update 1: Here's the code. I want to print the contents of the variable RxBufStr.

procedure ReceivingThread.OnExecute(AContext : TIdContext);
var
  RxBufStr: String;
begin    
  with AContext.Connection.IOHandler do
  begin
    CheckForDataOnSource(10);
    if not InputBufferIsEmpty then
    begin
      RxBufStr := InputBuffer.Extract();
    end;
  end;
end;

Answer 1

The data you have shown in the question looks like UTF-16 encoded data rather than UTF-8. However, since you are using a Unicode aware Delphi, and a string data type, clearly there has been an encoding mismatch. Your string variable appears to be double UTF-16 encoded if you can see what I mean!

It would appear therefore that InputBuffer.Extract is assuming that the data is transmitted using ANSI or UTF-8. In other words, an 8-bit encoding. But in fact the data is transmitted as UTF-16.

To solve the problem you need to align the reading of the buffer with the transmission of the buffer. You need to make sure that both sides use the same encoding. UTF-8 would be a good choice.

If the data in the buffer is UTF-16, then you can extract it with

RxBufStr := InputBuffer.Extract(-1, TIdTextEncoding.Unicode);

If you switch to UTF-8 then extract it with

RxBufStr := InputBuffer.Extract(-1, TIdTextEncoding.UTF8);

Answer 2

With

RxBufStr := InputBuffer.Extract();

the code does not specifiy a terminator or a data size, so it may happen that the client receives only a part of the sent data.

You can read the data with a given (known) length into a TIdBytes array and then convert it to a string using the correct encoding.

One way to do it is

TEncoding.Unicode.GetString( MyByteArray );

(found here)