Compatibility of a TStringList backup from old to newest versions

Question

I have a backup system which uses a TStringList, but I code with an old Delphi (Ansi strings).

Basically I have this when I save:

...
MyStringList.SaveToStream(Str);
StrSz := Str.Size;
MyBackupStream.Write(StrSz, SizeOf(Integer));
MyBackupStream.Write(Str.Memory^, StrSz);
...

And When I reload:

...
MyBackupStream.Read(StrSz, SizeOf(Integer));
Str.SetSize(StrSz);
MyBackupStream.Read(Str.Memory^, StrSz);
MyStringList.SetText := PChar( Str.Memory);
...

I use this sequential ( size + datasize bytes, then size + datasize bytes, etc) system for various component backup. In fact some stuffs are always 'read from' or 'written to' before the stringlist backup (I mean there are some data before and after the StringList backup).

Am I introducing a big problem here ( in case I switch to a modern Delphi version) ? Will the chunk still be castable in future delphi version ( in case I switch ?). Would it be necessary for me to write the string version in the backup header ?

Unfortunately I cannot test this. I think that if I least I write the string encoding type in the header I'll be able to cast it in the right way later, whatever is the Delphi version, won't I?

Answer 1

Use MyStringList.LoadFromStream(Str) instead of MyStringList.Text := PChar( Str.Memory).

First, your TStream data is not null-terminated, but using PChar the way you are requires a null terminator (you can use SetString() with a string variable to get around that).

Second, starting in D2009, String is now UnicodeString instead of AnsiString and PChar is now PWideChar instead of PAnsiChar. Your TStream data is Ansi instead of Unicode (even in D2009+ because SaveToStream() defaults to using TEncoding.Default, which is Ansi, for encoding the stream data), so casting the data to PWideChar will assign garbage to your TStringList.

In all versions, you should be using LoadFromStream(), but if you want to stick with setting the Text property then you need to do it like this, which works in all versions:

var
  ...
  S: AnsiString;
begin
  ...
  MyBackupStream.ReadBuffer(StrSz, SizeOf(Integer));
  Str.SetSize(StrSz);
  if StrSz > 0 then MyBackupStream.ReadBuffer(Str.Memory^, StrSz);
  SetString(S, PAnsiChar(Str.Memory), StrSz);
  MyStringList.Text := String(S);
  ...
end;

Or this:

var
  ...
  S: AnsiString;
begin
  ...
  MyBackupStream.ReadBuffer(StrSz, SizeOf(Integer));
  if StrSz > 0 then begin
    SetLength(S, StrSz);
    MyBackupStream.ReadBuffer(S[1], StrSz);
  end;
  MyStringList.Text := String(S);
  ...
end;

Or this:

var
  ...
  Str: TStringStream;
begin
  ...
  Str := TStringStream.Create;
  try
    MyBackupStream.ReadBuffer(StrSz, SizeOf(Integer));
    if StrSz > 0 then Str.CopyFrom(MyBackupStream, StrSz);
    MyStringList.Text := Str.DataString;
  finally
    Str.Free;
  end;
  ...
end;

Lastly, you should consider changing your stream data to use UTF-8 instead of Ansi for even better future compatibility. Both SaveToStream() and LoadFromStream() have an optional TEncoding parameter in D2009+, and UTF-8 is a lossless Unicode encoding whereas Ansi can loss data during Ansi/Unicode conversions. If your existing data is ASCII (no AnsiChar characters above #127), then UTF-8 is 100% backwards compatible with ASCII. But if the data is Ansi instead (has AnsiChar characters above #127), then you are best off changing your stream format in some way (add a header/version, etc) so you can differentiate between older and newer formats, then you can load older formats using Ansi, and save/load newer formats using Unicode/UTF-8.

Answer 2

I think you are on the right track. I remember, several years ago, I finished a similar task as you did. I had two sections for each chuck of data: header and content. Header contained information like starting address and length of the chunk. The content part contained actual data. This approach had never had any problem. In your case, the header only contained the size of the block. As to the version number of string, I recommend you to do so as, based on the release road of Delphi, it is very common that new releases are not backward compatible with old releases. Even if you don't have to use the version number later, it doesn't harm.