Why do I get pyodbc HY000 error if I commit() after complex SELECT in Vertica?

Question

I've been chasing down this error for a week:

pyodbc.Error: ('HY000', '[HY000] [Vertica][VerticaDSII] (20) An error occurred during query execution:  (20) (SQLFetch)')

After lots of trial and error, the key is a combination of this particular SELECT query (simpler ones don't error) plus a call to commit() after curs.execute(). I'm really curious to hear what you think is going on. Below (followed by configuration information) is the example. Any help in understanding why this is happening would be much appreciated. I'm happy to provide more detail if it would help.

Versions:

• Python: 3.3.2 [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux
• pyodbc: 3.0.7 (pyodbc.SQL_DBMS_VER: 18, pyodbc.SQL_DM_VER: 171, pyodbc.SQL_DRIVER_ODBC_VER: 77, pyodbc.SQL_DRIVER_VER: 7, pyodbc.SQL_ODBC_VER: 10)
• unixODBC: 2.2.14
• vertica: Vertica Analytic Database v7.0.0-0
• OS: Linux version 2.6.32-279.14.1.el6.x86_64 (mockbuild@c6b8.bsys.dev.centos.org) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) )

Source:

import pyodbc

query = '''
SELECT relVarTable0.id AS id, relVarTable1.val, relVarTable2.val, relVarTable3.val, relVarTable4.val
FROM (SELECT id FROM B) relVarTable0
LEFT JOIN
(SELECT B1.id AS id, AVG(A1.X) AS val
FROM B B1, AB AB1, A A1
WHERE B1.id = AB1.B_id AND A1.id = AB1.A_id AND A1.X IS NOT NULL
GROUP BY B1.id) relVarTable1
    ON relVarTable0.id = relVarTable1.id
LEFT JOIN
(SELECT B1.id AS id, B1.Y AS val
FROM B B1
WHERE B1.Y IS NOT NULL ) relVarTable2
    ON relVarTable0.id = relVarTable2.id
LEFT JOIN
(SELECT B1.id AS id, AVG(A1.W) AS val
FROM B B1, AB AB1, A A1
WHERE B1.id = AB1.B_id AND A1.id = AB1.A_id AND A1.W IS NOT NULL
GROUP BY B1.id) relVarTable3
    ON relVarTable0.id = relVarTable3.id
LEFT JOIN
(SELECT B1.id AS id, AVG(C1.Z) AS val
FROM B B1, BC BC1, C C1
WHERE B1.id = BC1.B_id AND C1.id = BC1.C_id AND C1.Z IS NOT NULL
GROUP BY B1.id) relVarTable4
    ON relVarTable0.id = relVarTable4.id;
'''

conn = pyodbc.connect('DSN=vertica_kdl_dsn;UserName=__;Password=__)
curs = conn.cursor()
curs.execute('SET search_path TO "$user", rcd_test, public, v_catalog, v_monitor, v_internal;')
curs.execute(query)
curs.commit()           # comment out to get no error
rows = curs.fetchall()  # if above commit called: pyodbc.Error: ('HY000', '[HY000] [Vertica][VerticaDSII] (20) An error occurred during query execution:  (20) (SQLFetch)')
curs.close()

Answer 1

FYI I got an easy solution over at the Vertica community forum:

add ResultBufferSize=0 in odbc.ini under the DSN [vertica_kdl_dsn] section

(The Hy000 was not a particularly helpful error message!)