Convert DOS line endings to Linux line endings in vim

StackOverflow https://stackoverflow.com/questions/82726

Question

If I open files I created in Windows, the lines all end with ^M.
How do I delete these characters all at once?

Was it helpful?

Solution

dos2unix is a commandline utility that will do this, or :%s/^M//g will if you use Ctrl-v Ctrl-m to input the ^M, or you can :set ff=unix and vim will do it for you.

Docs on the 'fileformat' setting are here, and the vim wiki has a comprehensive page on line ending conversions.

Alternately, if you move files back and forth a lot, you might not want to convert them, but rather to do :set ff=dos, so vim will know it's a DOS file and use DOS conventions for line endings.

OTHER TIPS

Change the line endings in the view:

:e ++ff=dos
:e ++ff=mac
:e ++ff=unix

This can also be used as saving operation (:w alone will not save using the line endings you see on screen):

:w ++ff=dos
:w ++ff=mac
:w ++ff=unix

And you can use it from the command-line:

for file in *.cpp
do 
    vi +':w ++ff=unix' +':q' "$file"
done

I typically use

:%s/\r/\r/g

which seems a little odd, but works because of the way that vim matches linefeeds. I also find it easier to remember :)

I prefer to use the following command :

:set fileformat=unix

You can also use mac or dos to respectively convert your file to macintosh or MS-DOS/MS-Windows file convention. And it does nothing if the file is already in the correct format.

For more information, see the vim help :

:help fileformat

:%s/\r+//g

In Vim, that strips all carriage returns, and leaves only newlines.

:set fileformat=unix to convert from dos to unix.

dos2unix can directly modify the file contents.

You can directly use it on the file, with no need for temporary file redirection.

dos2unix input.txt input.txt

The above uses the assumed US keyboard. Use the -437 option to use the UK keyboard.

dos2unix -437 input.txt input.txt

Convert directory of files from dos to Unix

Using command line and sed, find all files in current directory with the extension ".ext" and remove all "^M"

@ https://gist.github.com/sparkida/7773170

find $(pwd) -type f -name "*.ext" | while read file; do sed -e 's/^M//g' -i "$file"; done;

also, as mentioned above ^M = Ctrl+V + Ctrl+M (don't just type the caret "^" symbol and M)

tr -d '\15\32' < winfile.txt > unixfile.txt

(see: http://kb.iu.edu/data/acux.html)

Following steps can convert the file format for dos to unix:

:e ++ff=dos  Edit file again, using dos file format ('fileformats' is ignored).[A 1]
:setlocal ff=unix    This buffer will use LF-only line endings when written.[A 2]
:w   Write buffer using unix (LF-only) line endings.

Reference: http://vim.wikia.com/wiki/Change_end-of-line_format_for_dos-mac-unix

With the following command:

:%s/^M$//g

Get the ^M to appear type CtrlV then CtrlM. CtrlV tells Vim to take the next character entered literally.

I found a very easy way: Open the file with nano: nano file.txt

Press CTRL+O to save, but before pressing Enter, press: ALT+D to toggle betwen DOS and Unix/Linux line-endings, or: ALT+M to toggle betwen Mac and Unix/Linux line-endings then press Enter to save and CTRL+X to quit.

:g/Ctrl-v Ctrl-m/s///

CtrlM is the character \r, or carriage return, which DOS line endings add. CtrlV tells vim to insert a literal CtrlM character at the command line.

Taken as a whole, this command replaces all \r with nothing, removing them from the ends of lines.

You can use: 

vim somefile.txt +"%s/\r/\r/g" +wq

or dos2unix utility .

The comment about getting the ^M to appear is what worked for me. Merely typing "^M" in my vi got nothing (not found). The CTRL+V CTRL+M sequence did it perfectly though.

My working substitution command was

:%s/Ctrl-V Ctrl-M/\r/g

and it looked like this on my screen:

:%s/^M/\r/g

You can use the following command:
:%s/^V^M//g
where the '^' means use CTRL key.

below command is used for reformat all .sh file in current directory, I tested it on my Fedora OS.

for file in *.sh; do awk '{ sub("\r$", ""); print }' $file >luxubutmp; cp -f luxubutmp $file; rm -f luxubutmp ;done

Usually there is a dos2unix command you can use for this, just make sure you read the manual as the GNU and BSD versions differ on how they deal with the arguments.

BSD version:

dos2unix $FILENAME $FILENAME_OUT
mv $FILENAME_OUT $FILENAME

GNU version:

dos2unix $FILENAME

Alternatively, you can create your own dos2unix with any of the proposed answers here, for example:

function dos2unix(){
    [ "${!}" ] && [ -f "{$1}" ] || return 1;

    { echo ':set ff=unix';
      echo ':wq';
    } | vim "${1}";
}

I knew I'd seen this somewhere. Here is the FreeBSD login tip:

Need to remove all those ^M characters from a DOS file? Try

tr -d \\r < dosfile > newfile
    -- Originally by Dru <genesis@istar.ca>

In vim, type:

:w !dos2unix %

This will pipe the contents of your current buffer to the dos2unix command and write the results over the current contents. Vim will ask to reload the file after

To run directly into linux console: vim file.txt +"set ff=unix" +wq

Though this topic is very old, I'd like to put another stuff from wikia:

%s/\r\+$//g

that fill find all carriage return signs (one and more reps) up to the end of line and delete, so just \n will stay at eol.

This is my way. I opened a file in dos EOL and when I save the file that will automatically convert to unix EOL

autocmd BufWrite * :set ff=unix

if you create a file in NotePad or NotePad ++ in windows and bring it to Linux and open it by vim, you will see ^M at the end of each line. To remove this,

At your Linux terminal, type

dos2unix filename.ext

This will do the required magic.

I wanted newlines in place of the ^M's. Perl to the rescue:

perl -pi.bak -e 's/\x0d/\n/g' excel_created.txt

Or to write to stdout:

perl -p -e 's/\x0d/\n/g' < excel_created.txt
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top