On Windows, CR still here after read-line in Common Lisp

Question

I have a function reading a text file on Windows (thus a file with CRLF as line ending), and when I call read-line on this file, I get strings ending with CR, and this with SBCL or Clozure CL. With MKCL, both CR and LF are removed.

So I wonder, which implementation is right ?

According to the standard, The primary value, line, is the line that is read, represented as a string (without the trailing newline, if any). (see here). Thus, I think there should be no CR or LF left, but it's not very clear for me.

Of course, there are workarounds, but it's quite annoying, and I would like to know whether it's a bug, or simply implementation dependant.

Answer 1

I would say that SBCL and Clozure CL violate the Unicode Newline Guidelines:

Even if you know which characters represents NLF on your particular platform, on input and in interpretation, treat CR, LF, CRLF, and NEL the same. Only on output do you need to distinguish between them.

However, I think that trimming the return value is the wrong solution.

I think you need to use an implementation-specific :external-format argument to open the file with the correct (windows/CRLF) line termination mode.

Answer 2

CCL and SBCL are CL implementations with relatively weak Windows support. Both don't seem to recognize that CRLF is the line end format on Windows. Though you can debate with the maintainers - some might even think that this does not need to be supported.

The right thing would be to read a line and properly handle CRLF under Windows. Common Lisp has the idea that the line end character(s) are represented by a single NEWLINE character. NEWLINE best be mapped to CRLF on Windows. Several implementations are doing that right.

One way to work around that is to open the file with a special external format. CCL has that for example: http://ccl.clozure.com/manual/chapter4.5.html Then you can read the stream as usual without the need to trim characters.