How to determine the line ending of a file

Question

I have a bunch (hundreds) of files that are supposed to have Unix line endings. I strongly suspect that some of them have Windows line endings, and I want to programmatically figure out which ones do.

I know I can just run

flip -u

or something similar in a script to convert everything, but I want to be able to identify those files that need changing first.

Answer 1

You could use grep

egrep -l $'\r'\$ *

Answer 2

You can use the file tool, which will tell you the type of line ending. Or, you could just use dos2unix -U which will convert everything to Unix line endings, regardless of what it started with.

Answer 3

Something along the lines of:

perl -p -e 's[\r\n][WIN\n]; s[(?<!WIN)\n][UNIX\n]; s[\r][MAC\n];' FILENAME

though some of that regexp may need refining and tidying up.

That'll output your file with WIN, MAC, or UNIX at the end of each line. Good if your file is somehow a dreadful mess (or a diff) and has mixed endings.

Answer 4

Here's the most failsafe answer. Stimms answer doesn account for subdirectories and binary files

find . -type f -exec file {} \; | grep "CRLF" | awk -F ':' '{ print $1 }'

• Use file to find file type. Those with CRLF have windows return characters. The output of file is delimited by a :, and the first field is the path of the file.

Answer 5

Unix uses one byte, 0x0A (LineFeed), while windows uses two bytes, 0x0D 0x0A (Carriage Return, Line feed).

If you never see a 0x0D, then it's very likely Unix. If you see 0x0D 0x0A pairs then it's very likely MSDOS.

Answer 6

Windows use char 13 & 10 for line ending, unix only one of them ( i don't rememeber which one ). So you can replace char 13 & 10 for char 13 or 10 ( the one, which use unix ).

Answer 7

When you know which files has Windows line endings (0x0D 0x0A or \r \n), what you will do with that files? I supose, you will convert them into Unix line ends (0x0A or \n). You can convert file with Windows line endings into Unix line endings with sed utility, just use command:

$> sed -i 's/\r//' my_file_with_win_line_endings.txt

You can put it into script like this:

#!/bin/bash

function travers()
{
    for file in $(ls); do
        if [ -f "${file}" ]; then
            sed -i 's/\r//' "${file}"
        elif [ -d "${file}" ]; then
            cd "${file}"
            travers
            cd ..
        fi
    done
}

travers

If you run it from your root dir with files, at end you will be sure all files are with Unix line endings.