Question

edit : I have ripped out the lexer as it does not cleanly integrate with Qi and just obfuscates grammars (see here).


I need to give token types a name that I can use in on_error handler in qi. At the moment the _4 (qi::on_error<..>(...,std::cout << _4 << std::endl)) placeholder is giving me the regular expression string of the token. My lexer looks as follows :

template <typename Lexer>
struct tokens : lex::lexer<Lexer>
{
    tokens()
        : left_curly("\"{\""),
        right_curly("\"}\""),
        left_paren("\"(\""),
        right_paren("\")\""),
        colon(":"),
        scolon(";"),
        namespace_("(?i:namespace)"),
        event("(?i:event)"),
        optional("(?i:optional)"),
        required("(?i:required)"),
        repeated("(?i:repeated)"),
        t_int_4("(?i:int4)"),
        t_int_8("(?i:int8)"),
        t_string("(?i:string)"),
        ordinal("\\d+"),
        identifier("\\w+")

    {
        using boost::spirit::lex::_val;

        this->self
            = 
            left_curly    
            | right_curly 
            | left_paren
            | right_paren
            | colon         
            | scolon
            | namespace_      
            | event             
            | optional           
            | required          
            | repeated
            | t_int_4
            | t_int_8
            | t_string
            | ordinal             
            | identifier         
            | lex::token_def<>("[ \\t\\n]+")   [lex::_pass = lex::pass_flags::pass_ignore];
    }


    lex::token_def<lex::omit> left_curly, right_curly, colon, scolon,repeated, left_paren, right_paren;
    lex::token_def<lex::omit> namespace_, event, optional, required,t_int_4, t_int_8, t_string;
    lex::token_def<boost::uint32_t> ordinal;
    lex::token_def<std::string> identifier;
};
Was it helpful?

Solution

edit : I have ripped out the lexer as it does not cleanly integrate with Qi and just obfuscates grammars (see here).


Here is one brittle hack to detect what token was caught by the on_error mechanism. Wiring this up will not be pleasant :D

template<typename State>
struct error_handler_impl
{
    typedef void result_type;

    State & state;
    error_handler_impl(State & state) : state(state) {}

    template <typename What, typename Iterator>
    void operator()(const std::string & msg,What what, Iterator it) const
    {
        if(boost::get<boost::spirit::utf8_string>(what.value) == state.tok.identifier.definition())
        {
            std::cout << "token type is identifier" << std::endl;
        }
    }
};

state.tok is the lexer object, via which the token definitions may be accessed.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top