Optional token at end of file

Question

I am trying to write a grammar to parse a file where blank lines indicate the end of a block. I have grammar similar to this which almost works.

file : block+ EOF;
block : line+ NL;
line : stuff NL;

NL : '\r'? '\n';

This works except that the last block sometimes does not have an extra newline. Is there a good way to make the NL at the end of block optional when I am at the end of the file?

In antlr3, I would have done

block : line+ (NL | (EOF) => /* empty */ )

However, antlr4 does not have syntactic predicates, so I can't do that.

block : line+ NL? ;

should work, but then a block in the middle of the file could avoid its final newline. I don't think it will since a block can only be followed by a block. That means a block without the trailing newline followed by a block looks like one single block, and the parser will greedily combine them. However, it makes it less clear what the structure actually is. I can certainly imagine more complicated source file formats where this causes a problem.

Is there a good way to solve this?

Answer 1

Try something like this:

file : NL* block (NL+ block)* NL* EOF;
block : line (NL line)*;
line : stuff;

NL : '\r'? '\n';

Or simply append a line break at the end of your input.