Matlab: how to manage a lot of sparse data with the sparse command?

Question

The variable vector stores a multilinear function with 31 variables such that

>> tic; mlf=sparse(1,2^31)
toc
tic; mlf(1)=7
toc

mlf =

   All zero sparse: 1-by-2147483648

Elapsed time is 1.075814 seconds.

mlf =

                  (1,1)                       7

Elapsed time is 15.468432 seconds.

where it contains all possible terms in the multilinear function such as constant, $x_1$,x_2x_31$ and $x_30x_31$. Yet this initilization and particularly assignment take far too long time -- ~1 second and ~15 seconds here -- in reality each mlf has only about 1-20 terms so not even close to 2147483648! Now the times are far too large apparently due to too many extra zeros.

How to manage the large variable vector to store sparse information?

Answer 1

Shai commented here

"BTW, have you tried storing the mlf as a column vector rather than a row vector? sparse([],[],[],2^31, 1, 500);? If I'm not mistaken this should be easier to handle with Matlab's internal representation of sparse matrices."

and that did it!

>> tic;sparse([],[],[],2^31,1);toc 
Elapsed time is 0.549435 seconds.
>> tic;sparse([],[],[],1,2^31);toc 
Elapsed time is 15.102854 seconds.

Amazing!

Why is that so?

(If I'm allowed to so bluntly edit a post) Matlab's sparse matrices are stroed using three vectors: one that stores the row index of the non-zeros entries. The second stores the column index but in a compressed manner. Finally, the third vector stores the actual value of each entry.
The first and last vectors always have length as the number of non-zeros elements in the matrix. However, the compressed second one has length as the number of columns of the matrix regardless of the number of non-zero elements in the matrix.
Therefore, transposing mlf from 2^31 columns to 1 has a huge impact on the size of the second vector - and this is why the timing is so drastically affected.