building a lexical analyser using ml-lex

Question

I need to create a new instance of a lexer tied to the standard input stream.
However, when I type in

val lexer = makeLexer( fn n => inputLine( stdIn ) );

I get an error that I don't understand:

stdIn:1.5-11.13 Error: operator and operand don't agree [tycon mismatch]
  operator domain: int -> string
  operand:         int -> string option
  in expression:

(makeLexer is a function name present in my source code)

Answer 1

inputLine returns a string option, and my guess is a string is expected.

What you want to do is either have makeLexer take a string option, like so:

fun makeLexer  NONE    = <whatever you want to do when stream is empty>
  | makeLexer (SOME s) = <the normal body makeLexer, working on the string s>

or change your line to:

val lexer = makeLexer( fn n => valOf ( inputLine( stdIn ) ) );

valOf takes an option type and unpacks it.

Note that, since inputLine returns NONE when the stream is empty, it's probably a better idea to use the first approach, rather than the second.

Answer 2

An example of how to make an interactive stream is given on page 38 (or 32 in the paper) of the User's Guide to ML-Lex and ML-Yacc

The example code could be simpler by using inputLine. So I would use the example given by Sebastian, keeping in mind that inputLine might return NONE using stdIn atleast if the user presses CTRL-D.

val lexer =
let 
  fun input f =
      case TextIO.inputLine f of
        SOME s => s
      | NONE => raise Fail "Implement proper error handling."
in 
  Mlex.makeLexer (fn (n:int) => input TextIO.stdIn)
end

Also the calculator example on page 40 (34 in the paper) shows how to use this in a whole

In general the user guide contains some nice examples and explanations.