Insert into Star Schema efficiently using JDBC

Question

I have star schema model in which Server Table contains information about server name. Information Table contains information that I want for specific server. And Actual Data Table contains information about which server contains which information.

Server Table

Information Table

Actual Data Table

Now the problem that I am having is- I am trying to insert Data into the Data Table using JDBC. But I am unsure how should I add data into Actual Data Table in star schema model. Should I connect to database and insert it every time for each information or there is any direct way we can do that by communicating to database only one time. This is my code where I am getting all the information for each server. And IndexData is the class where I insert values into Oracle Database.

    public void fetchlog() {
            InputStream is = null;
            InputStream isUrl = null;
            FileOutputStream fos = null;
            try {
                is = HttpUtil.getFile(monitorUrl);
                if(monitorUrl.contains("stats.jsp") || monitorUrl.contains("monitor.jsp")) {
                    trimUrl = monitorUrl.replaceAll("(?<=/)(stats|monitor).jsp$", "ping");
                }
                isUrl = HttpUtil.getFile(trimUrl);
                BufferedReader in   = new BufferedReader (new InputStreamReader (is));
            String line;
            int i=0,j=0,k=0;
            while ((line = in.readLine()) != null) {
                if(line.contains("numDocs")) {
                    docs = in.readLine().trim();
    //So should I keep on inserting into Database for each information, like this
     //IndexData id = new IndexData(timeStamp, ServerName, InformationName, docs);
                }  else if(line.contains("indexSize")) {
                    indexSize = in.readLine().trim();
    //For this information-- the same way?
    //IndexData id = new IndexData(timeStamp, ServerName, InformationName, indexSize);
                } else if(line.contains("cumulative_lookups")) {
                    cacheHits= in.readLine().trim();
    //For this information too-- the same way?
    //IndexData id = new IndexData(timeStamp, ServerName, InformationName, cacheHits);
                } else if(line.contains("lastCommitTime")) {
                    lastCommitTime = in.readLine().trim();     
    //For this information too-- the same way?
    //IndexData id = new IndexData(timeStamp, ServerName, InformationName, lastCommitTime );

            }

            BufferedReader inUrl   = new BufferedReader (new InputStreamReader (isUrl));
            String lineUrl;
            Pattern regex = Pattern.compile("<str name=\"status\">(.*?)</str>");

            while ((lineUrl = inUrl.readLine()) != null) {
                System.out.println(lineUrl);
                if(lineUrl.contains("str name=\"status\"")) {
                    Matcher regexMatcher = regex.matcher(lineUrl);
                    if (regexMatcher.find()) {
                        upDown= regexMatcher.group(1);
//For this information too-- the same way?
    //IndexData id = new IndexData(timeStamp, ServerName, InformationName, upDown);

                    }                   
                     System.out.println("Status:- " + status);                  
                }  
            }
    //Or is there some other way we can insert directly into database by communicating with database only one time not multiple times for each information.     
            //IndexData id = new IndexData(timeStamp, ServerName, InformationName, Value);
                fos = new FileOutputStream(buildTargetPath());
                IOUtils.copy(is, fos);
            } catch (FileNotFoundException e) {
                log.error("File Exception in fetching monitor logs :" + e);
            } catch (IOException e) {
                log.error("Exception in fetching monitor logs :" + e);
            }
        }

I hope question is clear to everyone. Any suggestions will be appreciated.

Answer 1

There are two things I would suggest you look at. First, use a batch insert to perform all of the associated inserts in one JDBC transaction. For more information:

JDBC Batch Insert Example

I would also strongly recommend that you use a JDBC connection pooling library. We use c3p0 with our Postgres database. You can find more information here:

c3p0 Project Page

The basic idea would is to create a connection pool at startup time, then create JDBC batches for each set of related inserts.