Python: incorrect os.linesep on OS X

Question

Platform: OS X Mountain Lion
Python 2.7.4

I have a piece of script that needs to convert all eol characters in a string to unix style \n. I get a string supplied by Qsci.Scintilla.text() referenced to below as txt.

print 'original text: %s' % repr(unicode(txt))
print 'linesep: %s' % repr(os.linesep)
print 'fixed text: %s' % repr(unicode(txt).replace(os.linesep, '\n'))

This shows output like:

original text: u'exp.cnvs.show()\rself.sleep(1000)'
linesep: '\n'
fixed text: u'exp.cnvs.show()\rself.sleep(1000)'

All eol characters in there are reported as \r (which is correct for OS X af far as I know). Still os.linesep reports \n to be the eol character, causing the bottom statement to do nothing. The output of that statement of course should have been:

fixed text: u'exp.cnvs.show()\nself.sleep(1000)'

Does anyone know why os.linesep appears to report the wrong eol character on OSX?

Answer 1

No, \r (carriage return, or CR) is only correct for old Mac OS, so up to release 9.

OS X is a complete rewrite, and now a UNIX-based OS. It uses \n (line feed, or LF).

From Wikipedia on the subject of Newline:

• LF: Multics, Unix and Unix-like systems (GNU/Linux, Mac OS X, FreeBSD, AIX, Xenix, etc.), BeOS, Amiga, RISC OS and others.

[...]

• CR: Commodore 8-bit machines, Acorn BBC, ZX Spectrum, TRS-80, Apple II family, Mac OS up to version 9 and OS-9

See Historical reason behind different line ending at different platforms for some history behind this.

Instead, it's Scintilla that is in the wrong here. The Line endings documenation suggests it defaults to \r when on Mac:

Scintilla can interpret any of the Macintosh (\r), Unix (\n) and Windows (\r\n) line ends.

You can use the SCI_SETEOLMODE() function to alter that behaviour, whatever the Python binding equivalent is should be passed the os.linesep value to make Scintilla use the correct line separator for your platform.

I see that there is a QsciScintilla.setEolMode() method and a EOLMode enum to accomplish that; I guess you'll need to map python line ending variants to the enum and pass the right enum value to the .setEolMode() method.