Question

I tried to upload 3MB really small csv file with 30,000 rows and 4 columns . It took me more than an hour

cUrl = "jdbc:odbc:DSN; TYPE=FASTLOAD" ;
Connection conn = DriverManager.getConnection(cUrl, username, password);
String sql = "insert into Transactions(custID, transaction_date, amount, desc) values(?,?,?,?)";
PreparedStatement ps = conn.prepareStatement(sql);

    for ( ... ) 
    {
        ps.setString(1, custID);
        ps.setString(2, tran_date);
        ps.setString(3, amount);
        ps.setString(4, desc);
        ps.addBatch(); 
    }


    ps.executeBatch(); 

The addBatch runs very smoothly. Ones I get to the ps.executeBatch(); line it takes it forever . It took over an hour to upload a 3 MB csv file with 30,000 rows . Is it the way it is supposed to be

Was it helpful?

Solution

Loading 30.000 row should run in a few seconds as maximum.

You're still using the wrong connection string, there's no support for the FastLoad protocol in ODBC, TYPE=FASTLOAD is probably silently ignored.

What's the Primary Index of the Transactions table? A bad PI plus a SET table might result in slow loading.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top