idl strange symbols in file

Question

I've written several IDL programs to analyse some data. To keep it simple the programs read in some time varying data and calculate the fourier spectrum. This spectrum is written to file using this code:

openw,3,filename        
printf,3,[transpose(freq),transpose(power)],format='(e,e)'
close,3 

The file is then read by another program using this code:

rdfloat,filename,freq,power,/double

The rdfloat procedure can be found here: http://idlastro.gsfc.nasa.gov/

The error i get when trying to read the a file is: "Input conversion error. Unit: 101" When i delve in to the file being read, i notice several types of unrecognised characters. I dont know if these are a result of the writing to the file or some thing else related to the number of files being created (over 300 files)

These symbols/characters are in the place of a single number:

< dle> < dc1> < dc2> < dc3> < dc4> < can> < nak> < em> < soh> < syn>

Example of what appears in the file being read, Note they are not consecutive lines.

7.7346< dle>18165493007e+01   8.4796811549010105e+00
7.7354408697119453e+01   1.04459538071< dc2>1749e+01
7.7360701595839< can>28e+01   3.0447318983094189e+00

Whenever I run the procedures that write the files, there is always at least one file that has some or all of these characters. The file/s that contains these characters is always different.

Can anyone explain what these symbols are and what I might be doing to create them as well as how to ensure they are not written to file?

Answer 1

I see two things that may be causing a problem. But first, I want to suggest a few tips.

When you open a file, it is useful to use the /GET_LUN keyword because it allows IDL to find and use a logical unit number (LUN) that is available (e.g., in case you left LUN 3 open somewhere else). When you print formatted data, you should specify the total width and number of decimal places. It will make things easier because then you need not worry about changing spacings between numbers in a file.

So I would change your first set of code to the following (or some variant of the following):

OPENW,gunit,filename[0],/GET_LUN,ERROR=err
FOR j=0L, N_ELEMENTS(freq) - 1L DO BEGIN
  PRINTF,gunit,freq[j],power[j],FORMAT='(2e20.12)'
ENDFOR
FREE_LUN,gunit  ;;  this is better than using the CLOSE routine

So the first potential issue I see is that if your variable power was calculated using something like FFT.pro, then it will be a complex float or complex double, depending on the input and keywords used.

The second potential issue may be due to an incorrect format statement. You did not tell PRINTF how many columns or rows to expect. It might not know how to handle the input properly, so it guesses and may result in those characters you show. Those characters may be spacing characters due to the vague format statement or the software you are using to look at the files (e.g., I would not recommend using Word to open text files, use a text editor).

Side Note: You can open and read the file you just wrote in a similar fashion to what I showed above, but changed to the following:

n = FILE_LINES(filename[0])
freq = DBLARR(n)
power = DBLARR(n)
OPENR,gunit,filename[0],/GET_LUN,ERROR=err
FOR j=0L, N_ELEMENTS(freq) - 1L DO BEGIN
  READF,gunit,freq[j],power[j],FORMAT='(2e20.12)'
ENDFOR
FREE_LUN,gunit  ;;  this is better than using the CLOSE routine