Move the odbcConnect out of the per-record loop.
Increase the speed of inserting data frames to oracle tables in R using rodbc
Question
I am using RODBC to insert data frames to oracle table. I have Primary key set up on the table and if a duplicate data frame comes in it will oracle will reject it. In order to make sure I am not missing any data, first I try to insert the whole data frame, if any errors, I will try to insert each record at a time. But when I try to insert each record at a time, it is taking a very long time to finish. I was curious, has anybody done any type of work like this and what was the approach. My code is below:
tryCatch({
ch=odbcConnect("<id>",pwd = "<password>")
sqlSave(ch,dat, tablename="<tablename>", rownames=FALSE, append=TRUE)
},error = function(e)
{
print("unable to insert the data frame, will try by each row")
ch=odbcConnect("<id>",pwd = "<password>")
for (k in 1:nrow(dat))
{
j<-dat[k,]
tryCatch({
sqlSave(ch,j, tablename="<tablename>", rownames=FALSE, append=TRUE)
},error = function(e) {
print("unable to insert, duplicate values")
})
}
odbcClose(ch)
})
odbcClose(ch)
Solution
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow