How to automate dos2unix using shell script?

Question

I have a bunch of xml files in a directory that need to have the dos2unix command performed on them and new files will be added every so often. I Instead of manually performing dos2unix command on each files everytime I would like to automate it all with a script. I have never even looked at a shell script in my life but so far I have this from what I have read on a few tutorials:

FILES=/tmp/testFiles/*
for f in $FILES
do
  fname=`basename $f`
  dos2unix *.xml $f $fname
done

However I keep getting the 'usage' output showing up. I think the problem is that I am not assigning the name of the new file correctly (fname). Can anyone help.

Thanks, Alan

Answer 1

The reason you're getting a usage message is that dos2unix doesn't take the extra arguments you're supplying. It will, however, accept multiple filenames (also via globs). You don't need a loop unless you're processing more files than can be accepted on the command line.

dos2unix /tmp/testFiles/*.xml

Should be all you need, unless you need recursion:

find /tmp/testFiles -name '*.xml' -exec dos2unix {} +

(for GNU find)

Answer 2

If all files are in one directory (no recursion needed) then you're almost there.

for file in /tmp/testFiles/*.xml ; do
    dos2unix "$file"
done

By default dos2unix should convert in place and overwrite the original.

If recursion is needed you'll have to use find as well:

find /tmp/testFiles -name '*.xml' -print0 | while IFS= read -d '' file ; do
    dos2unix "$file"
done

Which will work on all files ending with .xml in /tmp/testFiles/ and all of its sub-directories.

If no other step are required you can skip the shell loop entirely:

Non-recursive:

find /tmp/testFiles -maxdepth 1 -name '*.xml' -exec dos2unix {} +

And for recursive:

find /tmp/testFiles -name '*.xml' -exec dos2unix {} +

In your original command I see you finding the base name of each file name and trying to pass that to dos2unix, but your intent is not clear. Later, in a comment, you say you just want to overwrite the files. My solution performs the conversion, creates no backups and overwrites the original with the converted version. I hope this was your intent.

Answer 3

mkdir /tmp/testFiles/converted/
for f in /tmp/testFiles/*.xml
do
  fname=`basename $f`
  dos2unix $f ${f/testFiles\//testFiles\/converted\/}
  # or for pure sh:
  # dos2unix $f $(echo $f | sed s@testFiles/@testFiles/converted/@)
done

The result will be saved in the converted/ subdirectory.

The construction ${f/testFiles\//testFiles\/converted\/} (thanks to Rush) or sed is used here to add converted/ before the name of the file:

$ echo /tmp/testFiles/1.xml | sed s@testFiles/@testFiles/converted/@
/tmp/testFiles/converted/1.xml

Answer 4

It is not clear which implementation of dos2unix you are using. Different implementations require different arguments. There are many different implementations around.

On RedHat/Fedora/Suse Linux you could just type

    dos2unix /tmp/testFiles/*.xml

On SunOS you are required to give an input and output file name, and the above command would destroy several of your files.

Which version are you using?

regards, Erwin