What can be reasons for `Error in .local(conn, statement, ...)´ in dbWriteTable from package MonetDB.R?

Question

I get the error, after I set up MonetDB and try to write a large data.frame as a new table in the default database (demo):

>dbWriteTable(conn, "table1", df)
Error in .local(conn, statement, ...) : 
   Unable to execute statement 'INSERT INTO table1 VALUES([...])

The data.frame has dimensions:

>dim(df)
[1] 148767    618

And has all columns formatted as character:

>all(lapply(df,class)=='character')
[1] TRUE

The error seems to stem from a string value being too long (Thanks @Hannes Mühleisen):

>dbGetException(conn)
$errNum
[1] 22001

$errMsg
[1] "value too long for type (var)char(255)"

How does MonetDB set upper bounds of new (VAR)CHAR variables (I did not find any info on upper bounds in the documentation)? Can a global upper bound be set or can the upper bound be set interactively when creating tables from R via MonetDB.R?

Answer 1

It might be a timeout issue (parameter to dbConnect()). If you'd like to debug, you can run

assignInNamespace("DEBUG_IO",TRUE,"MonetDB.R")

before connecting. If you post the output here, I can give you a better idea what could be going wrong. Finally, have you tried looking at the output of

dbGetException(conn)

?

UPDATE:

Sorry for that, the reason for this is that dbWriteTable uses the wrong SQL type for character data. At the moment, a VARCHAR(255) is used, which is limited to 255 characters indeed. What it should use is STRING, which has no limits. It is noted as a bug and will be fixed in the next release. Since this might take a while, here is a workaround: If you install from source, you may fix this by changing

setMethod("dbDataType", signature(dbObj="MonetDBConnection", obj = "ANY"), def=function(dbObj, obj, ...) {
        if (is.logical(obj)) "BOOLEAN"
        else if (is.integer(obj)) "INTEGER"
        else if (is.numeric(obj)) "DOUBLE PRECISION"
        else if (is.raw(obj)) "BLOB"

        else "VARCHAR(255)"
    }, valueClass = "character")

to

setMethod("dbDataType", signature(dbObj="MonetDBConnection", obj = "ANY"), def=function(dbObj, obj, ...) {
        if (is.logical(obj)) "BOOLEAN"
        else if (is.integer(obj)) "INTEGER"
        else if (is.numeric(obj)) "DOUBLE PRECISION"
        else if (is.raw(obj)) "BLOB"

        else "STRING"
    }, valueClass = "character")

in src/monetdb.R before installing the package using R CMD INSTALL. The R-forge builds will also be updated shortly, look for version 0.8.1 at https://r-forge.r-project.org/R/?group_id=1534