Why does C++11 still enforce lexical ordering for visibility

Question

For example, forward declarations and from the Wikipedia section on Alternative function syntax:

The type Ret is whatever the addition of types Lhs and Rhs will produce. Even with [...] decltype, this is not possible:

template<class Lhs, class Rhs>
decltype(lhs+rhs)  //Not legal C++11
      adding_func(const Lhs &lhs, const Rhs &rhs) {
  return lhs + rhs;
}

This is not legal C++ because lhs and rhs have not yet been defined; they will not be valid identifiers until after the parser has parsed the rest of the function prototype.

Back when system were under major memory pressure this was understandable (it allowed the passes to be done as streaming operations). But why, in this day and age (where my smart phone has enough RAM to hold the full source code, the full parse tree, and then some for any reasonable file) does the order of the tokens matter? Is there some strange corner case in the grammar that makes it impossible to, for example, find the end of the decltype non terminal production unless you know if an identifier in it is a Type or variable? And as another point, why does the order of global/namespace scoped declarations matter at all?

---

Edit, turns out this is legal:

class foo {};
foo Foober(int foo) { return ::foo(); }

If I ever run into this in the wild, it will end up on The Daily WTF, but it's still legal.

Answer 1

It is not a matter of capacity, but rather a matter of backward compatibility.

The new version of the language must preserve virtually all old functionality to maintain backward compatibility. In the original C++ the member function return type was not looked up in the scope of the class or function. It was looked up in the enclosing scope of function declaration. Changing this rule would be potentially disastrous for backward compatibility.

The only way around it is to make an exception specifically for decltype arguments. But this would be quite inconsistent and error prone.

Answer 2

The statement "this is not legal C++" is not quite accurate. It would be legal C++ were lhs and rhs defined in the scope in which the function declaration appears. The parameters lhs and rhs are not in the outer scope, but it is certainly possible that the names are in scope with some other declaration; changing the scope of the function return type could silently change the meaning of previously valid programs.

Scoping is not always in lexical order in C++. Inside a member function body, for example, all members are visible, even ones which have not yet been declared. (The leading return type is not part of a member function body either, but the trailing return is.)

Is there some strange corner case in the grammar that makes it impossible to, for example, find the end of the decltype non terminal production unless you know if an identifier in it is a Type or variable?

I don't think so; the parentheses around decltype's argument should be unambiguous. But if you don't know whether an identifier is a template or not, that can affect the meaning of <, > and >> tokens, in ways that can be really subtle. And not knowing whether an identifier is a type or a function also produces difficulties in parsing expressions (even in C). So there definitely are corner cases in which the "kind" of a name is important.

Answer 3

They have to draw the line between "inner" and "outer" scope somewhere. As you mentioned, it's easier to find the end of the production when the identifiers within it can be resolved. It's easier to implement in the compiler, and it's easier to consistently specify in the language standard. That is probably why the first compilers did it that way, and why the language always requires the typename and template keywords to resolve syntactic ambiguities caused by undefined (nested) identifiers in template expansion.

And an identifier can never be used before it appears in a declaration, which is a wonderful rule to help human or machine readers to find the relevant declaration. (Except between class members, where special two-pass parsing is applied to improve convenience.)

Once it's done one way, a commitment is made and the language can't change. Except by adding a feature, that is, and that's how the problem was solved by C++11.

template<class Lhs, class Rhs>
  auto adding_func(const Lhs &lhs, const Rhs &rhs) -> decltype(lhs+rhs)
    {return lhs + rhs;}

By the way, there's another workaround which they don't mention:

template<class Lhs, class Rhs>
  decltype( declval< Lhs >() + declval< Rhs >() )
    adding_func(const Lhs &lhs, const Rhs &rhs) {return lhs + rhs;}

Answer 4

Your premise is that forward declaration is purely for saving memory to allow for stream parsing (rather than holding the whole source file in memory) is not really correct.

A lot of the parsing algorithms we use are based on making a forward pass, and if you have to backtrack it can create an exponential explosion in parse time (or space) to disambiguate.

For example - to temporarily accept an identifier before it is declared without knowing anything about what it is - means that you have to proceed with the parse temporarily accounting for all the possibilities of what it is. (Including but not limited to a typo)