Read CSV file (Distinct Record) and write it to Oracle using Java - ORA-00001: unique constraint (DUMMY.SYS_C008271) violated

Question

I am totally new to Programming and Java. So when I ran my Java code, I got the following errors. I understand that my Oracle table has a unique constraint, so how do I read only a distinct value so when it gets written to the table, it does not give me a unique constraint error? When I tried to disable the constraint temporarily, everything works fine. But I am not supposed to alter the table. So what I need is to figure out how to read ONLY distinct records from CSV file and write it on the database.

PS: It is easy to do it on a database, here is my pseudocode: INSERT INTO TABLE_ORACLE (Col1, Col2, Col3, Col4) SELECT DISTINCT Col1, Col2, Col3, Col4 FROM TABLE_CSV

Thanks a lot!!

package dummy;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException; 
import java.util.Properties;
import java.io.*;

public class Main {
public static void main(String[] args) {

    String csvFile = "C:/Users/Jane/Documents/dummyfile.csv";
    BufferedReader br = null;
    String line = "";
    String csvSplitBy = ",";

    try {

        br = new BufferedReader(new FileReader(csvFile));
        for (; (line = br.readLine()) != null;) {
            while ((line = br.readLine()) != null) {
                // use comma as separator
                String[] data = line.split(csvSplitBy);
                if (data.length == 18) {
                    db_loader(data);
                }
            }
        }
    } 
    catch (Exception e) {
        e.printStackTrace();
    } 
    finally {
        if (br != null) {
            try {
                br.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    System.out.println("Done1");
}

public static void db_loader(String[] da) throws SQLException {
    String url = "jdbc:oracle:thin:@127.0.0.1:1521:xe";
    Properties props = new Properties();
    props.setProperty("user", "dummy999");
    props.setProperty("password", "dummy123");
    Connection conn = DriverManager.getConnection(url, props);
    PreparedStatement preStatement2 = conn
            .prepareStatement("insert into TABLE_ORACLE (col1, col2, col3, col4) values (?, ?, ?, ?)");
    for (int i = 0; i < da.length; i++) {
        preStatement2.setString(i + 1, da[i]);
    }
    preStatement2.executeQuery();
    preStatement2.close();
    try {
        Thread.sleep(100);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    System.out.println("done2");

}
}

Answer 1

You don't need both the for and while loops. I think you're trying to limit the number of updates to 18 per statement in load_db, but this won't work.

Instead, create a two-dimensional array with 18 rows and 4 columns(number of fields in CSV line).

Also, when we reach the last CSV line in br, i may be less than 18, so you need to execute db_load on the content of csv_data after exiting the read loop.

final int MAX_ROWS = 18;
final int CSV_FIELDS = 4;
int i=0;
String[][] csv_data = new String[MAX_ROWS][CSV_FIELDS];
while ((line = br.readLine()) != null) {
   // use comma as separator
   csv_data[i++] = line.split(csvSplitBy);
   if (i == 18) {
        db_loader(csv_data);
        i=0;
   }
}
if(i>0) db_loader(csv_data);

Answer 2

You can simply use a HashSet Which accepts unique records, and returns false when the record(key) already exists.

   Set<String> uniqueLines = null;

    try {

        uniqueLines   new HashSet<String>();
        br = new BufferedReader(new FileReader(csvFile));
        for (; (line = br.readLine()) != null;) {
            while ((line = br.readLine()) != null) {
               if(uniqueLines.add(line)) {// true only when it is unique record
                 // use comma as separator
                 String[] data = line.split(csvSplitBy);
                 if (data.length == 18) {
                     db_loader(data);
                 }
                }
            }
        }
    }