Unexpected Behaviour of GCC and VC++ Preprocessor

Question

I am trying to understand the C++ standard preprocessor requirement. A little bit tricky example I created has surprising results in GCC and VC++2010:

#define a(x,y) x##y
#define tzsW kka
a(t,zs  )W

GCC yields:

tzs W

Note the extra space added before W.

VC++2010 yields:

tzsW

Note there is no space added before W but the identifier is not further expanded. I scanned through the C++03 standard and cannot find anything saying we should prevent a new identifier (tzsW) from being created as in gcc. And nothing is about preventing this new identifier from further macro expanded (VC++ behaviour).

Why GCC and VC++2010 don't like the new identifier?

EDIT

if another macro invocation used, e.g.

a(t,zs  )[]

gcc yields:

tzs[]

Note no space added, showing gcc deliberately adds space to my previous case.

Answer 1

The output of the preprocessor is tokens, not plain text. Unless you use the token pasting operator, tokens are not normally combined in preprocessing.

When you view the output of the preprocessing step, you have to convert the tokens to text. gcc inserts the space so you are not mislead into thinking that tzsW is a single token. It doesn't need to do this in the tzs[ case as [ isn't a valid identifier character so there's no confusion.

Both compilers are correct in not treating tzsW as a single token to be re-expanded.

Note that the Visual C++ documentation acknowledges that compiling the output of the preprocessor may produce different and incorrect output compared with compiling the original source due to the fact that they don't insert whitespace to delimit tokens when converting the preprocessed output to text. This doesn't happen when the preprocessor output is passed straight to the next stage of the compiler in normal operation.

Answer 2

1. Space or no space?

   I did not found a reference in the standard, but this article discusses the issue. In short, it is not easy to decide if you want that space or not...

2. Replacing tzsW with kka or not?

   See 16.3.4 in the standard:

   After all parameters in the replacement list have been substituted, the resulting preprocessing token sequence is rescanned with all subsequent preprocessing tokens of the source file for more macro names to replace.

   This indicates that the behaviour of VC is not correct here.