Lexical analyzer output issue

Question

This is my lexical analyzer code when I enter as an input the following :

/*This is an example */

program
        var a,b:integer;

begin

        a =2;

        b =a+5;

        write(a);

        if b==1 then write(a);

end

the output must be like this :

<res,program>
<res,var> <id,a>,<id,b>:<res,integer>;
<res,begin>
<id,a> <assign,=><num,2>;
<id,b> <assign,=><id,a><addop,+><num,5>;
<res,write>(<id,a>);
<res,if> <id,b><relop,==><num,1> <res,then> <res,write>(<id,a>);
<res,end>

but I my output is :

Lexical Error~/hedor1>exampler < input\ .txt 
<res,program><res,var><id,a>,<id,b>:<res,integer>;<res,begin><id,a><assign,=><num,2>;<id,b><assign,=><id,a><addop,+><num,5>;<res,write>(<id,a>);<res,if><id,b><relop,==><num,1><res,then><res,write>(<id,a>);<res,end>

I don't know why it just avoids the newline and doesnot print it to the output although I have defined that in my patterns section \n printf("\n"); what is the problem?

Answer 1

Nowhere in your input do you have a single newline by itself. All you have are sequences of one or more whitespace characters (spaces, tabs and newlines). Since you have a rule that matches that, Flex uses the longest match.

Flex generates a greedy parser, which tries to match as much of the input as possible. For example, if it sees the input reality, it doesn't stop after matching real and then go on and match ity as a separate token. Instead, it matches all of reality.

In the same way, in your input after the starting comment you have not one but two newlines (since there is an empty line there), and this will be matched by your {whitespace}+ rule, instead of twice by the \n rule.