Delphi TClientDataSet, maximum number of fields per index

Question

I have a simple Delphi (2007) procedure that given a TDataSet and a (sub)list of fields returns a new TClientDataSet with the distinct values from the given TDataSet.

This works quite well.

In my proc I used the TClientDataSet index to populate the distinct values. It was fast and easy.

The problem is that TClientDataSet index support at maximum 16 fields. If you add more of them they will be silently ignored.

I need more than 16 fields in the dataset (and thus in the index).

Is there any solution? Some hack? Maybe some open source library to use as workaround?

I'm working offline so I must do it in memory. The size of the dataset is not huge

Answer 1

If you're needing to get distinct occurrences of records across more than 16 fields and you want to use an index to keep things fast you'll need to consider concatenating some of those fields. For example:

Test Field Field 1 Field 2 Field 3 Field 4 Apple~Banana~Carrot~Donut Apple Banana Carrot Donut

Create you index on the Test Field.

You might need to create multiple test fields if the total length of your other fields exceeds the maximum length of a text field.

Answer 2

You could swap out the TClientDataSet for a TjvCsvDataset from JVCL. It can be used as a pure "in memory dataset" replacement for Client Data Sets, without any need to read or write any CSV files on disk.

It is not quite like Client Data Set in design. I am not sure what benefit all these "Indexes" in a client data set offer you, other than that you can't have a field without an index definition, but in the case that this is all you need, you can set the TJvCsvDataSet.FieldDef property = 'Field1,Field2,.....FieldN' and then open the dataset and add as many rows as you like to the dataset. It is practically limited to the amount of memory you can address in a 32 bit process.