How to add a SQLite temp table from an R dataframe?

Question

I have an SQLite database connection to a database file. I want to extract some data from one of the tables, do some processing in R and then create a temporary table on the same connection from the processed data. It needs to be a temp table because users may not have write access to the database, but I want to be able to query this new data alongside the data already in the database.

so, for example:

require(sqldf)

db <- dbConnect(SQLite(), "tempdb")
dbWriteTable(db, "iris", iris)

# do some processing in R:
d <- dbGetQuery(db, "SELECT Petal_Length, Petal_Width FROM iris;")
names(d) <- c("length_2", "width_2")
d <- exp(d)

and then I want to make a temporary table in the connection db from d

I know I could do:

dbWriteTable(conn=db, name= "iris_proc", value = d)

but I need it in a temp table and there doesn't seem to be an option for this in dbWriteTable.

One workaround I thought of was to add a temp table and then add columns and update them:

dbGetQuery(db, "CREATE TEMP TABLE iris_proc AS SELECT Species FROM iris;")
dbGetQuery(db, "ALTER TABLE iris_proc ADD COLUMN length_2;")

But then I can't get the data from d into the columns:

dbGetQuery(db, paste("UPDATE iris2 SET length_2 =", paste(d$length_2, collapse = ", "),  ";"))
Error in sqliteExecStatement(con, statement, bind.data) :
    RS-DBI driver: (error in statement: near "4.05519996684467": syntax error)

I imagine that, even if I get this to work, it will be horribly inefficient.

I thought there might have been some way to do this with read.csv.sql but this does not seem to work with open connection objects.

Answer 1

Use an in-memory database for the temporary table:

library(RSQLite)

db <- dbConnect(SQLite(), "tempdb")
dbWriteTable(db, "iris", iris)
d <- dbGetQuery(db, "SELECT Petal_Length, Petal_Width FROM iris")
d <- exp(d)

dbGetQuery(db, "attach ':memory:' as mem")
dbWriteTable(db, "mem.d", d, row.names = FALSE) # d now in mem database

dbGetQuery(db, "select * from iris limit 3")
dbGetQuery(db, "select * from mem.d limit 3")

dbGetQuery(db, "select * from sqlite_master")
dbGetQuery(db, "select * from mem.sqlite_master")