Sqlite - store matrices in a table

Question

I am quite new to Sqlite and have a dilemma about database design. Suppose we have a number of matrices (of various sizes) that is going to be stored in a table. We can further assume that no matrise is sparse.

Let's say we have:

A = [[1, 4, 5], 
     [8, 1‚ 4], 
     [1, 1, 3]]

B = [['what', 'a', 'good', 'day'],
     ['for', 'a', 'walk', 'outside']]

C = [['AAA', 'BBB', 'CCC', 'DDD', 'EEE'],
     ['FFF', 'GGG', 'HHH', 'III', 'JJJ'],
     ['KKK', 'LLL', 'MMM', 'NNN', 'OOO']]


And D which is [NxM] 

When we create the table we do not know all the sizes that the matrices will have. I do not think it would be nice to alter the table size afterwards. What would be a recommended way to store the matrices to efficiently get them back? I wish to query out a matrix row-by-row.

I am thinking of transforming matrices into a column vector that somehow ends up in a table like this,

CREATE TABLE mat(id INT,
                 row INT,
                 col INT,
                 val TEXT)

How can I get them back line by line with a query in sqlite that looks like this for matrix A?

[1, 4, 5] 
[8, 1‚ 4]
[1, 1, 3]

Ideas? Or could someone kindly refer to any similar problems

---------------------- UPDATE ----------------------

Okay. My question was not clear enough. That is probably the way I'm intended to arrange the data in my database. I hope you can help me find a way to organize my database,

Suppose we have some sets of data:

Compilation    User    BogoMips
1              Andrew  1.04
1              Klaus   1.78
1              James   1.99
1              David   2.09
.              .       .
.              .       .
1              Alex    4.71



Compilation     Time    Temperature    Colour
2              10:20    10             Blue
2              10:28    21             Green
2              10:42    25             Red
.               .       .              .
.               .       .              .
2              18:16    16             Green



Compilation    Colour    Distance
3              Blue      4
3              Green     9
.              .         . 
.              .         .
3              Yellow    12

...And there will be many more sets of data with different numbers columns and new headers. Some header names will return in another set. In advance, we have no idea what kind of sets needs to be stored. Every set has a common header 'compilation' that binds them together.

How would you structure the data in a database?

I find it hard to believe that creating a new table for each set is a good solution. or?

My idea is to have two tables, headers and data.

CREATE TABLE headers (id INT,
                      header TEXT
                     )

CREATE TABLE data (id INT, 
                   compilation INT,
                   fk_header_id INT REFERENCES headers,
                   row INT,
                   col INT,
                   value TEXT)


So the populated tables looks like this,

SELECT * FROM headers;
id    header
------------
1     User
2     BogoMips
3     Time
4     Temperature
5     Colour
6     Distance


SELECT * FROM data;
id   compilation    fk_header_id   row   col   value
----------------------------------------------------
1    1              1              1      1    Andrew
2    1              2              1      2    1.04
3    1              1              2      1    Klaus
4    1              2              2      2    1.78
.    .              .              .      .    .
.    2              3              1      1    10:20
.    2              4              1      2    10
.    2              5              1      3    Blue
.    2              3              2      1    10:28
.    2              4              2      2    21
.    2              5              2      3    Green
.    .              .              .      .    .
.    3              5              1      1    Blue
.    3              6              1      2    4
.    3              5              2      1    Green
.    3              6              2      2    9
.    .              .              .      .    .
.
and so on

The problem is that I don't know how to query out the datasets in Sqlite. Anyone (Tony) have an idea?

Answer 1

You'd need a pivot / cross tab query (or it's join equivalent) to get the data out.

e.g

Select c1.value as col1, c2.value as col2, c3.value as col3
from data c1 on col = 1
inner join data c2 on col = 2 and c2.compilation = c1.compilation and c2.row = c1.row
inner join data c3 on col= 3 and c3.compilation = c1.compilation and c3.row = c1.row
Where c1.compilation = 1 order by c1.row

As you can see this is less than funny. In particular with the above, you'd have to know the number of columns in advance. Crosstab or pivot would relieve you of that in terms of the sql, but you'd still have to mess about to read in the data from the query result.

Haven't seen anything is your question that indicates a need to extract a row or a column from a matrix, never mind a cell from the db

My Table would start as simple as Compilation, Description, Matrix

Matrix would be sort of serialisation of a matrix object, Binary, xml even some sort of string eg. 1,2,3|4,5,6|7,8,9

If this was all I needed to store, I'd be looking at a NoSQL variant.