There are a number of ways to achieve, but based on you code, the simplest would be to ensure you are passing the GridBagConstraints
along with the component you are adding when you add them...
file_select_panel.add(file1_tf, gbc);
//...
file_select_panel.add(add_f1_button, gbc);
//...
file_select_panel.add(rem_f1_button, gbc);
//...etc...
You might find taking a closer look at How to Use GridBagLayout helps, in particular, take a look at the section labelled "Specifying Constraints"