Iterate over lines including blank lines

Question

Given a multiline string with some blank lines, how can I iterate over lines in Lua including the blank lines?

local s = "foo\nbar\n\njim"
for line in magiclines(s) do
  print( line=="" and "(blank)" or line)
end
--> foo
--> bar
--> (blank)
--> jim

This code does not include blank lines:

for line in string.gmatch(s,'[^\r\n]+') do print(line) end
--> foo
--> bar
--> jim

This code includes extra spurious blank lines:

for line in string.gmatch(s,"[^\r\n]*") do
  print( line=="" and "(blank)" or line)
end
--> foo
--> (blank)
--> bar
--> (blank)
--> (blank)
--> jim
--> (blank)

Answer 1

Try this:

function magiclines(s)
        if s:sub(-1)~="\n" then s=s.."\n" end
        return s:gmatch("(.-)\n")
end

Answer 2

Here’s a solution utilizing LPEG:

local lpeg      = require "lpeg"
local lpegmatch = lpeg.match
local P, C      = lpeg.P, lpeg.C

local iterlines
do
  local eol  = P"\r\n" + P"\n\r" + P"\n" + P"\r"
  local line = (1 - eol)^0
  iterlines = function (str, f)
    local lines = ((line / f) * eol)^0 * (line / f)
    return lpegmatch (lines, str)
  end
end

What you get is a function that can be used in place of an iterator. Its first argument is the string you want to iterate, the second is the action for each match:

--- print each line
iterlines ("foo\nbar\n\njim\n\r\r\nbaz\rfoo\n\nbuzz\n\n\n\n", print)

--- count lines while printf
local n = 0
iterlines ("foo\nbar\nbaz", function (line)
  n = n + 1
  io.write (string.format ("[%2d][%s]\n", n, line))
end)

Answer 3

Here is another lPeg solution because it seems I was writing it at the same time as phg. But since grammars are prettier, I'll still give it to you!

local lpeg = require "lpeg"
local C, V, P = lpeg.C, lpeg.V, lpeg.P

local g = P({ "S",
    S = (C(V("C")^0) * V("N"))^0 * C(V("C")^0),
    C = 1 - V("N"),
    N = P("\r\n") + "\n\r" + "\n" + "\r",
})

Use it like this:

local test = "Foo\n\nBar\rfoo\r\n\n\n\rbar"
for k,v in pairs({g:match(test)}) do
    print(">", v);
end

Or just print(g:match(test)) of course

Answer 4

See if this magiclines implementation suits your bill:

local function magiclines( str )
    local pos = 1;
    return function()
        if not pos then return nil end
        local  p1, p2 = string.find( str, "\r?\n", pos )
        local line
        if p1 then
            line = str:sub( pos, p1 - 1 )
            pos = p2 + 1
        else
            line = str:sub( pos )
            pos = nil
        end
        return line
    end
end

You can test it with the following code:

local text = [[
foo
bar

jim

woof
]]


for line in magiclines( text ) do
    print( line=="" and "(blank)" or line)
end

Output:

foo
bar
(blank)
jim
(blank)
woof
(blank)

Answer 5

The following pattern should match each line including blank lines with one caveat: the string must contain a terminating CR or LF.

local s = "foo\nbar\n\njim\n" -- added terminating \n

for line in s:gmatch("([^\r\n]*)[\r\n]") do
   print(line == "" and "(blank)" or line)
end

--> foo
--> bar
--> (blank)
--> jim

An alternate pattern that does not require a trailing CR or LF will produce a blank line as the last line (since is it acceptable to capture nothing).

local s = "foo\nbar\n\njim"

for line in s:gmatch("([^\r\n]*)[\r\n]?") do
   print(line == "" and "(blank)" or line)
end

--> foo
--> bar
--> (blank)
--> jim
--> (blank)

Answer 6

A bit optimized @lhf's answer:

function magiclines(s)
    return s:gmatch("(.-)$")
end