Read the files in a folder and then store each user id in a linkedhashset

Question

I have around 100 files in a folder. And I am trying to read all those files one by one. Each file will have data like this and each line resembles an user id.

960904056
6624084
1096552020
750160020
1776024
211592064
1044872088
166720020
1098616092
551384052
113184096
136704072

So I need to read that file line by line and then store each user id in a LinkedHashSet. I am able to read all the files from a particular folder with the below code. But with the below java code that I wrote, I am not sure how to read those files line by line and then store each user id in a LinkedHashSet?

public static void main(String args[]) {

    File folder = new File("C:\\userids-20130501");
    File[] listOfFiles = folder.listFiles();

    for (int i = 0; i < listOfFiles.length; i++) {
        File file = listOfFiles[i];
        if (file.isFile() && file.getName().endsWith(".txt")) {
            try {
                String content = FileUtils.readFileToString(file);
                System.out.println(content);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

Any help will be appreciated on this? And any better way to do the same process?

Answer 1

Since you are using FileUtils. The class has the method readLines() which returns a list of String.

You can then add this List of String to LinkedHashSet by using the addAll() method.

Try this -

public static void main(String args[]) {
File folder = new File("C:\\userids-20130501");
File[] listOfFiles = folder.listFiles();

Set<String> userIdSet = new LinkedHashSet<String>();
for (int i = 0; i < listOfFiles.length; i++) {
    File file = listOfFiles[i];
    if (file.isFile() && file.getName().endsWith(".txt")) {
        try {
            List<String> content = FileUtils.readLines(file, Charset.forName("UTF-8"));
            userIdSet.addAll(content);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 }

Answer 2

You can use the readLine() method of BufferedReader to read a file line by line as below:

public static void main(String args[]) {

File folder = new File("C:\\userids-20130501");
File[] listOfFiles = folder.listFiles();

Set<String> userIdSet = new LinkedHashSet<String>();
for (int i = 0; i < listOfFiles.length; i++) {
    File file = listOfFiles[i];
    if (file.isFile() && file.getName().endsWith(".txt")) {
        try {
          BufferedReader br = new BufferedReader(new FileReader(file));
          String line;
          while((line = br.readLine()) != null) {
              userIdSet.add(line);
           }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 }
}

Answer 3

If I understood you correctly then you can do something like below.

// Change generic if you want Integer
Set<String> userIdSet = new LinkedHashSet<String>(); 

// Your code


String content = FileUtils.readFileToString(file);
// Creating array by splitting for every new line content
String[] stn = content.split("\\n");
     for(String xx : stn){
        // parse it as Integer if you changed the generic
        // Adding the file content to Set
        userIdSet.add(xx); 
}

Answer 4

Here are a the things I would do different.

• Using an iterator for the files

• The new try statement allows opening up a “resource” in a try block and automatically closing the resource when the block is done. Ref

• Excluding invalid files with a continue statement. This removes one level of nesting. See this post.

All of these changes are highly opinionated of course.

    public static void main(String[] args) {

       Set<Integer> users = new LinkedHashSet<Integer>();

       File dir = new File("C:\\userids-20130501");
       for (File child : dir.listFiles()) {
         if (!child.isFile() || !child.getName().endsWith(".txt")) {
           continue;
         }

         try (BufferedReader in = new BufferedReader(
             new FileReader(child.getPath()))) {

           String id = null;
           while ((id = in.readLine()) != null) {
             users.add(Integer.parseInt(id));
           }
         } catch (IOException ex) {
           ex.printStackTrace();
         }
       }
   }