F# Specification Symbolic Operator VS Symbolic Keyword

Question

I am reading in the F# specification - the most recent one that I could find, found here - in an effort to learn the language in what is arguably the hard way. In section "3.6 Symbolic Keywords," the specification states that:

The following symbolic or partially symbolic character sequences are treated as keywords:

token symbolic-keyword =
    let! use! do! yield! return!
    | -> <- . : ( ) [ ] [< >] [| |] { }
    ' # :?> :? :> .. :: := ;; ; =
    _ ? ?? (*) <@ @> <@@ @@>

In the next section, "3.7 Symbolic Operators," it states that:

User-defined and library-defined symbolic operators are sequences of characters as shown below, except where the sequence of characters is a symbolic keyword (§3.6).

regexp first-op-char = !%&*+-./<=>@^|~
regexp op-char = first-op-char | ?
token quote-op-left =
    | <@ <@@
token quote-op-right =
    | @> @@>
token symbolic-op =
    | ?
    | ?<-
    | first-op-char op-char*
    | quote-op-left
    | quote-op-right

I might be missing something obvious, but it seems to me that the specification is stating that the operators/keywords ?, @>, @@>, <@, and <@@ are both symbolic keywords and symbolic operators. So... which one are they? How do I know weather to use a symbolic keyword token, or a symbolic operator token?

Thanks in advance, Brandon

EDIT To be clear, I want to know why the specification states that symbolic operators can be these symbols immediately after stating that they can't be.

Answer 1

The concept that helps to answer this question is keyword. A keyword is a reserved word. Also check your Dragon book.

A reserved word is basically a word that cannot be used by the programmer as an identifier, in this case a word such as if, or an operator such as ->. In other words if you try to use “let (if) =” or “let (->) =” you will get an error because you are using a reserved word.

For this question interpret Section 3.6 as, these are keywords reserved for use by F#. Section 3.7 as, you can create your own operators as long as they follow these rules and are not the same as one of the reserved keywords in section 3.6.

So if you want to create the operator --> you can, but you cannot create the operator ->

To answer your question "So... which one are they?" They are keywords and they are operators defined by the system, they cannot be used as user defined operators.

EDIT

Lets come at this from a different direction. All lexer rules are called by parser rules. It is easy to find uses of symbolic-op rule, i.e. op-name, but look for symbolic-keyword and you will find "such as 34.. are post-filtered to two tokens: one int and one symbolic-keyword, ―..‖." Helpful but it doesn't answer your question and now you are wondering why is there a lexer rule not called by a parser rule. I don't know, this is a spec and not a formal grammar definition. If you take a look at the F# source you may find that the spec and grammar do not match up as you would like. In other words, the spec is meant to help you understand the language, I would not use it as a definitive set of rules for building a compiler.

Putting spec in terms or a car, the specs tell you what you can expect from the car, how the car should function, or the limits for a parameter, it does not tell you how to build the car.

If I were writing a compiler I would interpret the case of 34.. to mean, don't create tokens for symbolic-keywords on the first pass, but use post processing to filter them out into appropriate tokens. In other words, rewrite the token stream in a second pass. If it were me I would verify that the spec is a complete grammar before using it to build a compiler. If however you want to press on, then I would probably skip looking for symbolic-keywords on the first pass, and use stream rewriting after the token stream is created on the first pass.

If you want to understand more about token filtering in F# see section 15 of the spec, "Lexical Filtering"; it gives a descent explanation of how light syntax is converted to regular syntax by rewriting the token stream.