Modifying the Shunting Yard Algorithm (c++)

Question

I have a shunting yard algorithm in proper working order, but I noticed a special quirk:

1 + ( 3 * ( 4 + 5 ) )

parses correctly to

1 3 4 5 + * +,

but

1 + (3 * (4 + 5))

fails, and parses to

1 * + 5)) +

I want to get it to parse the second problem properly, so that the result is the same as the first. How can I accomplish this?

Notes: I derived my algorithm from wikipedia: http://en.wikipedia.org/wiki/Shunting-yard_algorithm#The_algorithm_in_detail

My algorithm code is:

string switchingYard(string input)
{
stringstream io(input);
ProcessStack switch_stack;
vector<string> out;
while(io.good())
{
    string token;
    io >> token;
    if(isdigit(token[0]) || (token[0] == '.' && isdigit(token[1]))
        || ( (token[0] == '-' && isdigit(token[1])) || (token[0] == '-' && token[1] == '.' && isdigit(token[2])) ) )
    {
        out.push_back(token);
    }


    if(isFunctionToken(token))
    {
        switch_stack.pushNode(token);
    }

    if(isArgSeparator(token[0]))
    {
        bool mismatch_parens = false;
        do{
            if(switch_stack.length() == 1)
            {
                if(switch_stack.peekChar() != '(')
                {
                    mismatch_parens = true;
                    break;
                }
            }
            string opPop = switch_stack.popNode();
            out.push_back(opPop);
        }while(switch_stack.peekChar() != '(');
        if(mismatch_parens)
            return "MISMATCH_ERROR";
    }

    if(isOperator(token[0]))
    {
        while(  isOperator(switch_stack.peekChar()) &&
                ((left_assoc(token[0]) && (op_preced(token[0]) == op_preced(switch_stack.peekChar()) )) || (op_preced(token[0]) < op_preced(switch_stack.peekChar())) ) )
        {
            string popped = switch_stack.popNode();
            out.push_back(popped);
        }
        switch_stack.pushNode(token);
    }

    if(token == "(")
        switch_stack.pushNode(token);

    if(token == ")")
    {
        bool mismatch_parens = false;
        while(switch_stack.peekChar() != '(')
        {
            if(switch_stack.length() == 0 || (switch_stack.length() == 1 && switch_stack.peekChar() != '('))
            {
                mismatch_parens = true;
                break;
            }
            string opPop = switch_stack.popNode();
            out.push_back(opPop);
        }
        if(mismatch_parens)
            return "MISMATCH_ERROR";
        string parensPop = switch_stack.popNode();
        if(isFunctionToken(switch_stack.peek()))
        {
            string funcPop = switch_stack.popNode();
            out.push_back(funcPop);
        }

    }
}
while(switch_stack.length() > 0)
{
    if(switch_stack.peekChar() == '(' || switch_stack.peekChar() == ')')
        return "MISMATCH_ERROR";
    string opPop = switch_stack.popNode();
    out.push_back(opPop);
}
string ret;
for(int i = 0; (unsigned)i < out.size(); i++)
{
    ret += out[i];
    if((unsigned)i < out.size()-1)
        ret += " ";
}
cout << "returning:\n" << ret << endl;
return ret;
}

Edit: Ok, so I just got an idea. So when the parser encounters the '(3' token, it would otherwise treat the two characters as one, and discard the whole thing, but what if I called the function recursively, passing in the substring of the input string which starts at the '3' character? I would then only need to add the shunted string to the output vector, and call ignore on the stringstream! I'm talking about making these changes:

string switchingYard(string input)

becomes

string switchingYard(string input, int depth)

and

if((token[0] == '(' || token[0] == ')') && (isdigit(token[1]) || token[1] == '.' || isOperator(token[1]))
            {
                string shunted_recur = out.push_back(switchingYard(input.substr(io.tellg()+1),depth+1));
            }

gets added to the end of the while loop. Thoughts?

Answer 1

Your problem is that the parser reads strings with:

io >> token;

The simple solution, in my opinion, would be to simply read a character at a time. E.g.

char ch;

io >> ch;

I would actually write a function that reads a token - it would know things like "a sequence of digits is a number" and separate out operators, parenthesis, etc. It would return an object (class or structure type) that holds an "type of element" and "value (if relevant) - so it could be type "number" and value "4711", or type "operator", value "+".

You will need a "state" for your tokeniser, which will include a "lookahead" character (one char should be enough), so that you can stop when you have gone past the end of a number, and then pick up the character that "stopped being a number" next time around.