What is best way to implement variable length arrays?

Question

I want to store a large result set from database in memory. Every record has variable length and access time must be as fast as arrays. What is the best way to implement this? I was thinking of keeping offsets in a separate table and storing all of the records consecutively? Is it odd? (Programming Language: Delphi)

Answer 1

Not sure I totally follow you, but have a look at TList.

In Delphi 7 at least, it is implemented as an arrary of pointers. You can use the capacity property to pre allocate the list ahead of time if you know how many results are coming back.

The list will automatically grow if it runs out of space. How much it grows by depends on how big the list is.

Take a look at the source for the classes unit to see what it's doing.

Edit: Also in D2009 genric support was added to TList which makes it a bit nicer to use.

Answer 2

The best way is probably to contain an array of pointers to records. You won't have to deal with offsets, in that case, and lookups will be constant time.

Answer 3

Why not use a MEMORY version of your database? Most have a way to keep a complete table in memory, usually involving the SQL keyword MEMORY. You'd copy the table from disk to the memory table, and then can use all the normal database operations at memory speed. I know this works well in DBISAM.

Answer 4

Following mj2008, you could use a TCLientDataset instead of a record array. How large is that resultset?

Answer 5

I'd use TList, and store pointers to your record.

type
  pMyRecord : ^TMyRecord;
...
...
...
var
  p : pMyRecord;
...
...
New(p);
with p^ do
begin
  ...
  ...
end;
...
MyList.Add(P);