an antlr4 parser in C# --- seems right but does not work

Question

grammar CSVParser;
//

table   :   line+;

line    :   NAME
',' PEAK ',' 
STARTYEAR ',' 
ENDYEAR ',' LENGTH NEWLINE
;

NEWLINE :   '\r'? '\n'
        ;   

NAME    :   ('"'(~'"')*'"') 
;

PEAK    :   ([0-9]+);

STARTYEAR   :   ([0-9]+);

ENDYEAR :   ([0-9]+);

LENGTH  :   [0-9]+;

As you can see, i'd like to parse a CSV table like this:

"ANNUAL REVIEW OF IMMUNOLOGY, VOL 31",0,0,1,1
"",0,0,1,1
"CA-A CANCER JOURNAL FOR CLINICIANS",1,1,2,1
"NATURE CHEMISTRY",1,1,3,2
"NATURE PHOTONICS",1,1,3,2
"ANNUAL REVIEW OF IMMUNOLOGY, VOL 30",1,1,2,1
"PHYSICS TODAY",2,1,3,2
"NATURE BIOTECHNOLOGY",2,2,4,2
"CHEMICAL SOCIETY REVIEWS",2,1,3,2
"NATURE REVIEWS GENETICS",2,2,3,1

but there are exceptions:

line 1:40 mismatched input '0' expecting STARTYEAR
line 2:5 mismatched input '0' expecting STARTYEAR
line 3:39 mismatched input '1' expecting STARTYEAR
line 4:21 mismatched input '1' expecting STARTYEAR
line 5:21 mismatched input '1' expecting STARTYEAR
line 6:40 mismatched input '1' expecting STARTYEAR
line 7:18 mismatched input '1' expecting STARTYEAR
line 8:25 mismatched input '2' expecting STARTYEAR
line 9:29 mismatched input '1' expecting STARTYEAR
line 10:28 mismatched input '2' expecting STARTYEAR
line 11:31 mismatched input '2' expecting STARTYEAR
line 12:42 mismatched input '2' expecting STARTYEAR
line 13:19 mismatched input '1' expecting STARTYEAR
line 14:40 mismatched input '2' expecting STARTYEAR
line 15:34 mismatched input '2' expecting STARTYEAR
line 16:29 mismatched input '2' expecting STARTYEAR
line 17:40 mismatched input '2' expecting STARTYEAR
line 18:40 mismatched input '2' expecting STARTYEAR
line 19:43 mismatched input '2' expecting STARTYEAR
line 20:40 mismatched input '3' expecting STARTYEAR

What's going wrong?

Oh, Stackoverflow tells me to add more details. But I think the code here are sufficient enough, as the csv file is shown here.

Answer 1

Many of your lexer rules match the same. When [0-9]+ is matched, only a PEAK will ever be created and never a STARTYEAR, ENDYEAR or LENGTH. The lexer does not create tokens based on what the parser "needs".

Do something like this instead:

grammar CSVParser;

table     :   line+ EOF;
line      :   NAME ',' peak ',' startyear ',' endyear ',' length NEWLINE;
peak      :   NUMBER;    
startyear :   NUMBER;
endyear   :   NUMBER;    
length    :   NUMBER;

NEWLINE   :   '\r'? '\n';   
NAME      :   '"' (~'"')* '"';
NUMBER    :   [0-9]+;