sed: replacing end of line

Question

I need to quote end of line and using this solution to replace end of line which working perfectly when file has multiple lines.

Problem arises when I try to use this method with files that have no \n or have single line ending with \n. In this case sed does not replace anything, even second s/ command does not work. Also looks like last end of line is never replaced.

My command is:

sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' <file>

(replace \n to \\n and replace a to _ for example).

I use hexdump -C to display files here:

There are tests I made:

$ # Test1. single line without \n
$ echo -n 'abc' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  61 62 63                                          |abc|
# but expected "_bc"

$ # Test2. single line with \n
$ echo 'abc' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  61 62 63 0a                                       |abc.|
# but expected "_bc\n"

$ # Test3. two lines
$ echo -e 'abc\n' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  5f 62 63 5c 6e 0a                                 |_bc\n.|
# but expected "_bc\n\n"

Questions: Why second s/ does not replace anything in Test1 and Test2? Is there any method to fix replacing of s/\n/\\n/ and s/a/_/g in all this tests?

P.S.: I don't want some workaround like adding newline at end of stream before sed processing and removing it after.

EDIT: looks like N command does not read single line, even if it followed by '\n'. Any idea how to fix it?

EDIT2: Seems like this is expected behavior of N command. Is there any method on how to replace last line ending?

Answer 1

Sad, but I don't find any method to prevent sed from adding new line to output when N command is used. I didn't want to some workaround to like add new line before processing and removing it after, but seems like there is no other method.

This is the final solution with workaround:

sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' <<< "$VAR" | tr -d '\n'

<<< like echo adds new line into input string, tr removes it.

With that line all test are OK:

$ echo -e "abc" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
00000000  5f 62 63                                          |_bc|

$ echo -e "abc\n" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
00000000  5f 62 63 5c 6e                                    |_bc\n|

$ echo -e "abc\n\n" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
00000000  5f 62 63 5c 6e 5c 6e                              |_bc\n\n|

P.S.: thanks potong for figuring out the :a;$!{N;ba}.

P.P.S.: If someone will give working answer without tr, I will re-accept it.

Answer 2

This might work for you (GNU sed):

sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' file

or:

sed ':a;$!N;$!ba;s/\n/\\n/g;y/a/_/' file

Logically it makes sense: one cannot get the next line if you are on the last line.

Answer 3

Can you use perl instead?

$ echo -n "abc" | perl -pe 's/\n$/\\n/;' -pe 's/a/_a/;' | od -x
0000000 615f 6362
0000004
$ echo "abc" | perl -pe 's/\n$/\\n/;' -pe 's/a/_a/;' | od -x
0000000 615f 6362 6e5c
0000006
$ echo -e 'abc\n' | perl -pe 's/\n$/\\n/;' -pe 's/a/_a/;' | od -x
0000000 615f 6362 6e5c 6e5c
0000010