How to inspect GCC preprocessor defines at specific lines in source code

Question

Is there a way to dump all current preprocessor defines at some point in the source file? Or another way to inspect changes in preprocessor directives between two points in the source file?

I did not find anything here. Here's an example to give the idea:

#define FOO

#pragma message "defines before whatever.h"
#pragma please_dump_all_defines
#include <whatever.h>
#pragma message "defines after whatever.h"
#pragma please_dump_all_defines

// rest of the file

A different way a way to get the information would work too, for example with gcc -E somehow, as long as it takes into account that FOO above may affect what exactly is defined by the include file, and can track multiple #define/#undef, etc.

Answer 1

Certainly the GCC preprocessor can't do exactly what you want but it does have a -dCHARS option, for flag combinations CHARS, that you can leverage with a little scripting to extract the changes in preprocessor definitions between two points in a translation unit.

I'll illustrate with a translation unit comprising these two files:

foo.c

#define B 1
#define C 2
#pragma message Begin
#include "bar.h"
#pragma message End
#undef B
#undef C
#define B 4
#define C 5 

bar.h

#ifndef BAR_H
#define BAR_H

#undef B
#undef C
#ifdef A
#define B 2
#define C 3
#endif

#endif

Invoke:

cpp -dD foo.c

The -dD option preserves the #define|#undef directives in the otherwise preprocessed output. There are > 500 lines of that output, so I'll just cite the interesting tail:

# 1 "<command-line>" 2
# 1 "foo.c"
#define B 1
#define C 2

# 3 "foo.c"
#pragma message Begin
# 3 "foo.c"

# 1 "bar.h" 1

#define BAR_H 

#undef B
#undef C
# 5 "foo.c" 2

# 5 "foo.c"
#pragma message End
# 5 "foo.c"

#undef B
#undef C
#define B 4
#define C 5

Alternatively, invoke:

cpp -dD -DA foo.c

and the corresponding tail (with my comments) is:

# 1 "<command-line>" 2
# 1 "foo.c"
#define B 1
#define C 2

# 3 "foo.c"
#pragma message Begin
# 3 "foo.c"

# 1 "bar.h" 1

#define BAR_H 

#undef B
#undef C

#define B 2     //<- New with -DA
#define C 3     //<- New with -DA
# 5 "foo.c" 2

# 5 "foo.c"
#pragma message End
# 5 "foo.c"

#undef B
#undef C
#define B 4
#define C 5

What a script has to do with this is:

• 1) Extract the #define|#undef directives that precede the start-point marker, #pragma message Begin, retaining only the rolling latest per macro name.
• 2) Repeat 1) from the start-point to the end-point marker #pragma message End.
• 3) Match the two sets by macro name and report the before/after differences.

As your luck would have it, it happens that I need to blow the cobwebs off my (never very lustrous) python for an interview, so here's a script (only cursorily debugged):

macrodiff.py

#!/usr/bin/python

import sys, argparse, os, string, re, subprocess, shlex
from subprocess import call, CalledProcessError

class macro_directive:
    def __init__(self,directive = None,name = None,definition = None):
        self.__directive = directive
        self.__name = name
        self.__definition = definition
    def __eq__(self,other):
        return  self.__name == other.__name and \
                self.__directive == other.__directive and \
                self.__definition == other.__definition
    def __neq__(self,other):
        return not __eq__(self,other)
    @property
    def empty(self):
        return not self.__directive
    @property
    def directive(self):
        return self.__directive
    @property
    def name(self):
        return self.__name
    @property
    def definition(self):
        return self.__definition

    @property
    def desc(self):
        desc = self.__directive + ' ' + self.__name
        if self.__definition:
            desc += ' '
            desc += self.__definition
        return desc

    @staticmethod
    def read(line):
        match = re.match('^\s*#\s*(define|undef)\s+(\w+) \s*(.*)$',line)
        if match:
            directive = match.group(1)
            name = match.group(2)
            if directive == 'define':
                return macro_directive(directive,name,match.group(3))
            else:
                return macro_directive(directive,name)
        else:
            return macro_directive()
    @staticmethod
    def make_dict(lines):
        d = {}
        for line in lines:
            md = macro_directive.read(line);
            if not md.empty:
                d[md.name] = md
        return d


def find_marker(lines,marker):
    for i, line in enumerate(lines):
        if line.find(marker) == 0:
            return i;
    return -1

def split_by_marker(lines,marker):
    mark_i = find_marker(lines,marker)
    if mark_i != -1:
        return [lines[:mark_i],lines[mark_i:]]
    return [[],lines]

parser = argparse.ArgumentParser(
    prog="macrodiff",
    formatter_class=argparse.RawDescriptionHelpFormatter,
    description='Extract changes in simple preprocessor macro values between' +
        ' two marked points in a C/C++ translation unit. ' +
        'Function-like macros are not supported')
parser.add_argument('-s', '--start', metavar='STARTSTR',required=True,
    help='The initial macro values will be those in effect when the' +
        ' first line commencing with STARTSTR is read')
parser.add_argument('-e', '--end', metavar='ENDSTR',
    help='The final macro values will be those in effect when the first line' +
    ' commencing with ENDSTR is read, or if --end is not given then those' +
    ' in effect at end-of-file ')
parser.add_argument('--pp', default='cpp -dD',metavar='PP',
    help='PP is the preprocessor command to invoke. Default \'cpp -dD\'')
parser.add_argument('--ppflags',default='',metavar='PPFLAGS',
    help='PPFLAGS are additional options to be passed to PP')
parser.add_argument('infile',metavar='FILE',nargs=1,
    help='FILE is a C/C++ source file to be processed')

args = vars(parser.parse_args())
startstr = args['start'];
endstr = args['end'];
stdout = ''
command = args['pp'] + ' ' + args['ppflags'] + ' ' + args['infile'][0]

try:
    stdout = subprocess.check_output(shlex.split(command))
except CalledProcessError, e:
    sys.stderr.write( '***Error: Command \"' + command + '\" failed: \"' + \
        e.output + '\": ' + 'syscode = ' + str(e.returncode) + '\n')
    sys.exit(e.returncode)
lines = stdout.splitlines();
lines_before,lines_after = split_by_marker(lines,startstr);
if not lines_before:
    sys.stderr.write( '***Error: STARTSTR \"' + startstr + '\" not found\n')
    sys.exit(1)
if endstr:
    lines_after, ignore = split_by_marker(lines_after,endstr);
    if not lines_after:
        sys.stderr.write( '***Error: ENDSTR \"' + endstr + '\" not found\n')
        sys.exit(1)

directives_dict_before = macro_directive.make_dict(lines_before)
directives_dict_after = macro_directive.make_dict(lines_after)
intersection = \
    directives_dict_before.viewkeys() & directives_dict_after.viewkeys()

for key in intersection:
    before = directives_dict_before[key]
    after = directives_dict_after[key]
    if before != after:
        print 'BEFORE[' + before.desc + '] AFTER[' + after.desc +']'  

sys.exit(0)

Usage

$ ./macrodiff.py -h
usage: macrodiff [-h] -s STARTSTR [-e ENDSTR] [--pp PP] [--ppflags PPFLAGS]
                    FILE

Extract changes in simple preprocessor macro values between two marked points in a C/C++ translation unit. Function-like macros are not supported

positional arguments:
  FILE                  FILE is a C/C++ source file to be processed

optional arguments:
  -h, --help            show this help message and exit
  -s STARTSTR, --start STARTSTR
                        The initial macro values will be those in effect when
                        the first line commencing with STARTSTR is read
  -e ENDSTR, --end ENDSTR
                        The final macro values will be those in effect when
                        the first line commencing with ENDSTR is read, or if
                        --end is not given then those in effect at end-of-file
  --pp PP               PP is the preprocessor command to invoke. Default 'cpp
                        -dD'
  --ppflags PPFLAGS     PPFLAGS are additional options to be passed to PP

Try this with:

$ ./macrodiff.py -s='#pragma message Begin' -e='#pragma message End' foo.c

output:

BEFORE[define C 2] AFTER[undef C]
BEFORE[define B 1] AFTER[undef B]

or:

$ ./macrodiff.py -s='#pragma message Begin' -e='#pragma message End' --ppflags='-DA' foo.c

output:

BEFORE[define C 2] AFTER[define C 3]
BEFORE[define B 1] AFTER[define B 2]

or:

$ ./macrodiff.py -s='#pragma message Begin' --ppflags='-DA' foo.c

This time the differences are taken between #pragma message Begin and end-of-file. Output:

BEFORE[define C 2] AFTER[define C 5]
BEFORE[define B 1] AFTER[define B 4]

If you'd rather use distinctive comments rather than pragmas as the start and end markers, then add -C to the PPFLAGS. That will retain comments in the preprocessed output, except those in directives.

Support for function-like macros is left as an exercise.

Answer 2

I'm pretty sure you should be able to code something like this up using Boost Wave.

And certainly with libclang, though that would both be a lot more complex and incur way more dependencies.

Let me whether I can whip up something based on Boost Wave.