Providing lookup list from R vector as SQL table for RODBC lookup

Question

I have a list of IDs in an R vector.

IDlist <- c(23, 232, 434, 35445)

I would like to write an RODBC sqlQuery with a clause stating something like

WHERE idname IN IDlist

Do I have to read the whole table and then merge it to the idList vector within R? Or how can I provide these values to the RODBC statement, so recover only the records I'm interested in?

Note: As the list is quite long, pasting individual values into the SQL statement, as in the answer below, won't do it.

Answer 1

You could always construct the statement using paste

IDlist <- c(23, 232, 434, 35445)
paste("WHERE idname IN (", paste(IDlist, collapse = ", "), ")")
#[1] "WHERE idname IN ( 23, 232, 434, 35445 )"

Clearly you would need to add more to this to construct your exact statement

Answer 2

I put together a solution to a similar problem by combining the tips here and here and running in batches. Approximate code follows (retyped from an isolated machine):

#assuming you have a list of IDs you want to match in vIDs and an RODBC connection in mycon

#queries that don't change
q_create_tmp <- "create table #tmptbl (ID int)"
q_get_records <- "select * from mastertbl as X join #tmptbl as Y on (X.ID = Y.ID)"
q_del_tmp <- "drop table #tmptbl"

#initialize counters and storage
start_row <- 1
batch_size <- 1000
allresults <- data.frame()

while(start_row <= length(vIDs) {
    end_row <- min(length(vIDs), start_row+batch_size-1)
    q_fill_tmp <- sprintf("insert into #tmptbl (ID) values %s", paste(sprintf("(%d)", vIDs[start_row:end_row]), collapse=","))

    q_all <- list(q_create_tmp, q_fill_tmp, q_get_records, q_del_tmp)
    sqlOutput <- lapply(q_all, function(x) sqlQuery(mycon, x))

    allresults <- rbind(allresults, sqlOutput[[3]])
    start_row <- end_row + 1
}