Storing Unicode data to Database

Question

Currently we are migrating from Delphi 2007 to Delphi XE2 to support Unicode . We store a large XML data as blobs in the database. To insert the blob value into the database field we use the code below

Param.DataType := ftBlob;
Param.AsBlob := Value;

But now to support unicode stuff, we conveted all the blob fields in the database to nvarchar (max). We changed the above code to

Param.DataType := ftwideString;
Param.AsString := Value;

Is this the right thing to do, or should it be handled differently so as to store Unicode data into the database.

Answer 1

BLOBs will store whatever you throw at them without inferring anything about it. Although they may have little manipulation capabilities, they will never modify the data. On the other hands, a character based field (even a CLOB), will have an associated character set. Thereby data stored there will be converted back and forth from the source character set and the destination one. For example if your XML file is UTF-8 encoded and the field is UTF-16 it will be converted when written and read. Of course if both encodings are the same no conversion is performed. If your database lacks an XML type, I would store them in binary lobs if they could be in different encodings and no operations upon them (but writing/reading) are needed. If they are always in the same encoding a character lob could be easier to work with. Using a character field with a large XML file can lead to errors due to the max field size.

Answer 2

Though your solution should work, I would suggest to keep blob field format, use UTF8 as default encoding for database (firebird example):

CREATE DATABASE 'localhost:c:\Data\MyDatabase.FDB' user 'SYSDBA' password 'masterkey'
  DEFAULT CHARACTER SET UTF8;

and convert your XML string to UTF8 encoding prior to saving it as blob.