EBNF grammar (ANTLR)

Question

I've got a problem with EBNF grammar in ANTLRWorks:

line 37:    
upper_lower_case
: LOWER_CASE 
| UPPER_CASE
;

line 42:
CLASSNAME 
:   UPPER_CASE (DIGITS | upper_lower_case )*
;

line 51:
UPPER_CASE 
:   'A'..'Z'
;

line 55:
LOWER_CASE 
:   'a'..'z'
;

line 60:
DIGITS  :   '0'..'9'
;

I want CLASSNAME to always start with the capital letter and than it can consists of digits, upper or lower case letters.

Error log:

[13:11:59] warning(200): classgenerator.g:43:42: 
Decision can match input such as "'0'..'9'" using multiple alternatives: 1, 2

As a result, alternative(s) 2 were disabled for that input
[13:11:59] warning(200): classgenerator.g:43:42: 
Decision can match input such as "<EOT>" using multiple alternatives: 2, 3

As a result, alternative(s) 3 were disabled for that input
[13:11:59] error(201): classgenerator.g:43:42: The following alternatives can never be 
matched: 3

[13:11:59] error(208): classgenerator.g:60:1: The following token definitions can never 
be matched because prior tokens match the same input: UPPER_CASE,DIGITS

Could anyone help me solve this problem? Thanks in advance.

Regards, Hladeo

EDIT:

So I should use fragment keyword if it doesn't refers to the tokens? In this way using fragment keyword will be wrong?

tokens {
PUBLIC    = '+';
PRIVATE   = '-';
PROTECTED = '=';
}

fragment ACCESSOR
: PUBLIC
| PRIVATE
| PROTECTED
;

and another question.

OBJECTNAME  
:   UPPER_LOWER_CASE (UPPER_LOWER_CASE | DIGIT)*
;

OBJECTNAME should consists of at least one letter (upper or lower cased doesn't matter) and optionally of another letters or digits - what's wrong with that part of the code? When I try to type for example variable - it's okay, but when I start with capital letter Variable I'm getting an error:

line 1:15 mismatched input 'Variable' expecting OBJECTNAME

Answer 1

Your lexer rule CLASSNAME currently references parser rule upper_lower_case (lexer rules start with an uppercase letter; parser rules start with lowercase). Lexer rules can only reference lexer rules.

In addition, it appears that UPPER_CASE, LOWER_CASE, and DIGITS should not create tokens themselves so they should be marked as fragment rules. In the following example, I changed DIGITS to DIGIT since it only ever matches one digit.

CLASSNAME : UPPER_CASE (DIGIT | UPPER_LOWER_CASE)*;

fragment UPPER_LOWER_CASE : LOWER_CASE | UPPER_CASE;
fragment UPPER_CASE : 'A'..'Z';
fragment LOWER_CASE : 'a'..'z';
fragment DIGIT : '0'..'9';

Edit 1 (for the edits in the question):

• A piece of text in the input can only have one token type. For example, consider the input text X3. Since this text could match a CLASSNAME or an OBJECTNAME, the lexer will end up assigning it the type of the first rule appearing in the grammar. In other words, if CLASSNAME appears before OBJECTNAME in the grammar, the input X3 will always be a CLASSNAME token and will never be a OBJECTNAME token. If OBJECTNAME appears before CLASSNAME in the grammar, the input X3 will always be an OBJECTNAME and never be a CLASSNAME (in fact, in this case no token will ever be a CLASSNAME).

• Your ACCESSOR rule looks like it should be a parser rule, like the following:

  accessor : PUBLIC | PROTECTED | PRIVATE;
  

Edit 2 (for the comment about distinguishing CLASSNAME and OBJECTNAME):

To distinguish between CLASSNAME and OBJECTNAME, you can create a lexer rule IDENTIFIER which matches either.

IDENTIFIER : UPPER_LOWER_CASE (DIGIT | UPPER_LOWER_CASE)*;

You can then create a parser rule to handle the distinction:

classname : IDENTIFIER;
objectname : IDENTIFIER;

Obviously this allows x3 to be a classname, which is not valid in your language. When possible, I always prefer to relax the parser rules a bit and perform further validation later where I can provide a better error message. For example, if you allow x3 to match classname, then after you parse the input and have an AST (ANTLR 3) or parse tree (ANTLR 4), you can look for all instances of classname and make sure that the matched IDENTIFIER starts with the required upper case letter.

Example error message produced by the parser's automatic error reporting:

line 1:15 mismatched input 'variable' expecting CLASSNAME

Example error message produced by separate validation:

line 1:15 class name variable must start with an upper case letter