Create gist index on cube array with 512 dimensions

Question

I want to perform nearest neighbors search on db with 5000 entries of cube array with 512 dimensions.

I referred to this answer and used docker image of postgresql with cube array limit set to 2048 which can be found in the comment to the answer. But I'm getting the following error:

ERROR: failed to add item to index page in "faq_c_idx" SQL state: XX000

It works with dimensions=~200, but when I set higher dimensions I keep receiving the error above.

Here's the query:

CREATE TABLE mytable
AS
  SELECT i, cube(array_agg(random()::float)) AS c
  FROM generate_series(1,5000) AS i
  CROSS JOIN LATERAL generate_series(1,512)
    AS x
  GROUP BY i;

CREATE INDEX ON mytable USING gist ( c );
ANALYZE mytable;

Answer 1

It can't find a place to store the index tuple, probably because the tuple is larger than a page and so can't fit on any page. You could recompile with a larger blocksize, e.g. ./configure --with-blocksize=32, but then you will just run into a different limitation, probably INDEX_SIZE_MASK, which is not so easy to work around.

So, you just can't do this. But if you could, an index into a 512 dimensional cube is not likely to be all that useful anyway. It will probably be slower than a seq-scan would be, or at least not faster by very much.