How to increase index row maximum size in Postgresql?

Question

I want to perform KNN on 512-dimensional vector using the solution suggested here.

1. I couldn't create table with 512-d cube array. Solved by changing the CUBE_MAX_DIM in the source code.
2. Couldn't create index on table with dimensions > 200. Set block_size=32. Now works with dimensionality<=510.
3. Got new error after changing block_size and trying to create index on table with 512-d vectors ERROR: index row requires 8208 bytes, maximum size is 8191

Is it possible to increase this limit?

Answer 1

That would require significant changes, and I doubt it can be done easily.

See these excerpts from src/include/access/itup.h:

/*
 * Index tuple header structure
 *
 * All index tuples start with IndexTupleData.  If the HasNulls bit is set,
 * this is followed by an IndexAttributeBitMapData.  The index attribute
 * values follow, beginning at a MAXALIGN boundary.
 *
 * Note that the space allocated for the bitmap does not vary with the number
 * of attributes; that is because we don't have room to store the number of
 * attributes in the header.  Given the MAXALIGN constraint there's no space
 * savings to be had anyway, for usual values of INDEX_MAX_KEYS.
 */

typedef struct IndexTupleData
{
    ItemPointerData t_tid;      /* reference TID to heap tuple */

    /* ---------------
     * t_info is laid out in the following fashion:
     *
     * 15th (high) bit: has nulls
     * 14th bit: has var-width attributes
     * 13th bit: AM-defined meaning
     * 12-0 bit: size of tuple
     * ---------------
     */

    unsigned short t_info;      /* various info about tuple */

} IndexTupleData;               /* MORE DATA FOLLOWS AT END OF STRUCT */
[...]

/*
 * t_info manipulation macros
 */
#define INDEX_SIZE_MASK 0x1FFF
#define INDEX_AM_RESERVED_BIT 0x2000    /* reserved for index-AM specific
                                         * usage */
#define INDEX_VAR_MASK  0x4000
#define INDEX_NULL_MASK 0x8000

The limit you are hitting is INDEX_SIZE_MASK, and to increase it, you'd have to change the tuple header so that t_info has more than two bytes.

Perhaps it is as simple as that, but it might have repercussions in other parts of the code.