Recursively remove trailing characters

Question

I just copied couple of files from windows to unix and they all have ^M at the end. I know how to remove them using vi, but I can only do one file at a time, is there a way I can do it for all the files in the folder?. There are like 60 files and manually doing it for all of them is time consuming!.

I'm open to using other tools as well!

PS: The OS is Solaris

Thanks

Answer 1

For posterity, let's post the solution from within VI. You can remove the Ctrl-M at the end of every line like this:

:%s/^V^M$//

Note that this is what you type, wnere ^V means Ctrl-V and ^M means Ctrl-M. The idea here is that ^V will "escape" the following ^M, so that you can match it in the substitution regex.

And the % expression means "do this on every line".

Note that this may or may not work in vim, depending on your settings.

But your question asks how to do this in vi, in which you can't easily make a change to multiple files. If you're open to using other tools, please indicate so in your question.

You can use sed on a single file or stream:

$ printf 'one\r\ntwo\r\n' > /tmp/test.txt
$ od -c < /tmp/test.txt
0000000    o   n   e  \r  \n   t   w   o  \r  \n                        
0000012
$ sed -i'' -e 's/^M$//' /tmp/test.txt 
$ od -c < /tmp/test.txt
0000000    o   n   e  \n   t   w   o  \n                                
0000010
$ 

In this case, in /bin/sh in FreeBSD, I escaped the ^M by ... you guessed it ... using ^V.

When using sed's -i option, you can specify multiple files and they will all be modified in place, perhaps eliminating the need to wrap this in a script. If you want to put this into a script anyway, I recommend you try to do so, and then ask for help if it doesn't work. That's the StackOverflow Way. :-)

Or just use Jonathan's for loop example. You don't need temp files.

UPDATE

If your sed does not have a -i option, then you can still do this pretty easily in a for loop:

[ghoti@pc ~]$ od -c /tmp/test1.txt
0000000    o   n   e  \r  \n   t   w   o  \r  \n                        
0000012
[ghoti@pc ~]$ for f in /tmp/test*.txt; do sed -e 's/^M$//' "$f" > /tmp/temp.$$ && mv -v /tmp/temp.$$ "$f"; done
/tmp/temp.26687 -> /tmp/test1.txt
/tmp/temp.26687 -> /tmp/test2.txt
[ghoti@pc ~]$ od -c /tmp/test1.txt
0000000    o   n   e  \n   t   w   o  \n                                
0000010

Answer 2

If you don't have a dos2unix or dtou command on your machine, you can use tr instead:

for file in "$@" # LIst of files passed as argument to script
do
    tr -d '\015' < "$file" > tmp.$$
    cp tmp.$$ "$file"
done
rm tmp.$$

You can add trap commands around that to clean up if you interrupt. Using cp instead of mv preserves owner, permissions, symlinks, hard links.

Answer 3

use the command dos2ux.

dos2ux file >file2