How to turn an Indy UTF-8 response into a native Delphi (Unicode)String?

Question

Using Indy THTTP I obtain a response that has Content-Type: text/html; charset=UTF-8 and store it in a TStringStream. If I then use ReponseStream.ReadString(ResponseStream.Size), the resulting String is not correctly shown. I bet this is due to the fact that Windows uses UTF-16.

I tried a few things with TEncoding.UTF8 and TEncoding.Convert that only messed up the result even more (started to look Chinese).

Here's the current code:

var
  LHTTP: TIdHTTP;
  LResponseStream: TStringStream;
  LResponse: String;
begin
  LResponseStream := TStringStream.Create();
  try
    LHTTP := TIdHTTP.Create(nil);
    try
      LHTTP.Get('url', LResponseStream); // Returns 'hęllo'
    finally
      LHTTP.Free;
    end;
    LResponseStream.Position := 0;
    LResponse := LResponseStream.ReadString(LResponseStream.Size);
    ShowMessage(LResponse); // Make me pretty
  finally
    LResponseStream.Free;
  end;
end;

What should I change to get a regular Delphi String...?

Answer 1

TIdHTTP has an overloaded version of Get() that returns a String. It will decode the UTF-8 into UTF-16 for you:

LResponse := LHTTP.Get('url');

Answer 2

If the content you are trying to download is encoded as UTF-8 character set, you could simply force TStringStream to re-encode that data to UTF-8 internally in this way :

LResponseStream := TStringStream.Create('', TEncoding.UTF8);