bash deletes my file while looping “~$ ./program > file.dat” (gnuplot)

Question

I'm doing some script on bash to plot data with gnuplot. But it doesn't work.

I have a program which throws data to the command line. I collect that data on a file and then I loop this process over to get the desired result. Say I have this script

 N = 10
 ./program.exe > data_$N.dat # Creating the data to be plotted
 for ((i = 0 ; i<1 ; i++)) do # Dummy loop to throw all together to gnuplot
  echo "set size square"
  echo "set xrange[-$N-1:$N+1]; set yrange[-$N-1:$N+1];"
  echo "plot '-' using 1:2"
  for((j = 0 ; j <= 9 ; j++)) do
   # Throwing the data to gnuplot
   echo "cat data_$N.dat" 
   # Updating the file and overwriting on it
   echo "cat data_$N.dat | xargs ./program.exe > data_$N.dat" | bash  
   echo "e" 
   echo "pause 0.5"
  done
 done | gnuplot -persist # Throwing the formatted data to gnuplot

So, basically I throw formatted text to gnuplot from a file, changing the content of it, but without changing the name of the file.

Of course, I don't wanna change the name of the file because I make the calculations corresponding to that step with that file, and then I overwrite the file with the updated data. Kind of a buffer. The computational flow would be

./program.exe > data.dat # Making the initial data
begin gnuplot loop 
 throw data to gnuplot
 "cat data.dat | xargs ./program.exe > data.dat" # update data overwriting file
end gnuplot loop
pipe all the script to gnuplot

Hope is clear and you can help me :). Thanks!

PD. I'm a physicist.

Answer 1

Redirection with > is done by the shell. So the first thing that happens when executing cat data_$N.dat | xargs ./program.exe > data_$N.dat is that the shell opens data_$N.dat for write and truncates it. Now it's empty, so when cat starts it finds a file but it has already been truncated.

Just redirect your output to a temporary location (it's common to use $$ in that which returns your PID so your scripts are somewhat independent). So ... > tmp_$$.dat and then right after that you can mv -f tmp_$$.dat data_$N.dat.