using scons for literate programming

Question

Using noweb, I would either like to generate a document file (or a source file) from a noweb input file **.nw

From hand I would do something like that:

notangle my_program.nw > my_program.cpp
g++ -c my_program.o my_program.cpp
ln -o myapp ... my_program.o ...

Now I like to ask whether I can use scons to automate this.

Imagine, my project directory is on $MYPROJECT. THere we have "$MYPROJECT/SConstruct". Now I defined a scons tool "tangle.py" (simplified from "noweb.py). Here we have "$MYPROJECT/site_scons/site_tools/tangle.py"

import SCons.Builder

def cpp_emit (target,source, env):
    # I dont know what to do here ... please help
    return (target,source)

# Tangle to .cpp
__noweb_tangle_builder = SCons.Builder.Builder(
    action='/usr/bin/notangle $SOURCES >$TARGET',
    suffix='.cpp',
    src_suffix='.nw',
    emitter=cpp_emit)

# -----------------------
def generate(env):
    env['BUILDERS']['tangle']= __noweb_tangle_builder

def exists(env):
    return 1

This tool generates a cpp-file from a nw-file.

But if I do something like

def cpp_emit (target,source, env):
    new_source=target[0].name
    new_target=new_source.rstrip(".cpp")+".o"
    target.append(new_target)
    source.append(new_source)
    return (target, source)

I get into a dependency circle. SCons will find and abort with an error message.

Doing ...

def cpp_emit (target,source, env):
    new_source=target[0].name

    # someprogram.cpp -> someprogram.o
    new_target=new_source.rstrip(".cpp")+".o" 

    # lets avoid dependency cycle
    t = [] 
    t.append(new_target)
    source.append(new_source)
    # oops, we dropped target test.cpp. It wont be generated.
    return (t, source) 

... the tool would stop generating a cpp file from a nw file. (Cpp target dropped)

Do you know a working way to do use scons for literate programming?

thank you for reading.

---

Leonard

Answer 1

Here is the tool that I created. Note the use of env['BUILDERS']['Object'].src_builder to allow env.Program() to accept noweb files.

# site_cons/site_tools/tangle.py
import SCons.Builder

__all__=['generate', 'exists']

tangle_builder = SCons.Builder.Builder(
    action='$NOTANGLE $SOURCES > $TARGET',
    suffix = '.cpp',
    src_suffix = '.nw')

def generate(env):
    env['NOTANGLE'] = exists(env)
    env['BUILDERS']['Tangle'] = tangle_builder
    if 'Object' in env['BUILDERS']:
        env['BUILDERS']['Object'].src_builder.append('Tangle')

def exists(env):
    if 'NOTANGLE' in env:
        return env['NOTANGLE']
    return env.WhereIs('notangle')

And its use:

# SConstruct
env = Environment(tools=['default', 'tangle'])
env.Program('my_program.nw')

Here is the output of the above SConstruct:

$ scons -Q
/usr/bin/notangle my_program.nw > my_program.cpp
g++ -o my_program.o -c my_program.cpp
g++ -o my_program my_program.o
$ scons -c
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Cleaning targets ...
Removed my_program.cpp
Removed my_program.o
Removed my_program
scons: done cleaning targets.

Answer 2

Seems like you're trying to add the object file without actually compiling the cpp file.

I made a small example that should help clear up the situation. Basically, since you configured the suffix, and src_suffix in the call to Builder, the sources and targets are correctly setup by SCons, and you dont need the emitter.

def cpp_emit (target,source, env):
    for t in target:
        print 'Emitter target: %s' % (t)
    for s in source:
        print 'Emitter source: %s' % (s.name)

    return (target,source)

# Tangle to .cpp
builder = Builder(
    action='/home/notroot/projects/sandbox/Emitter/builder.sh $SOURCES $TARGET',
    suffix='.cc',
    src_suffix='.nw',
    emitter=cpp_emit)

env = Environment()
env['BUILDERS']['tangle'] = builder

tangleTarget = env.tangle(target='main.cc', source='main.nw')
env.Object(source=tangleTarget)

And here is the output:

$ scons
scons: Reading SConscript files ...
Emitter target: main.cc
Emitter source: main.nw
scons: done reading SConscript files.
scons: Building targets ...
/home/notroot/projects/sandbox/Emitter/builder.sh main.nw main.cc
g++ -o main.o -c main.cc
scons: done building targets.

$ scons -c
scons: Reading SConscript files ...
Emitter target: main.cc
Emitter source: main.nw
scons: done reading SConscript files.
scons: Cleaning targets ...
Removed main.cc
Removed main.o
scons: done cleaning targets.

I did the following to get the Builder to generate the cc file and compile it, but it doesnt clean the object file.

import os

def cpp_emit(target,source, env):
    for s in source:
        print 'Emitter source: %s' % (s.name)
    for t in target:
        print 'Emitter target: %s' % (t)

    return (target,source)

def build_function(target, source, env):
    # Code to build "target" from "source"
    for t in target:
        print 'Builder target: %s' % (t.name)
    for s in source:
        print 'Builder source: %s' % (s.name)

    buildStr='/home/notroot/projects/sandbox/Emitter/builder.sh %s %s' % (source[0].name, target[0].name)
    os.system(buildStr)

    trgt = env.Object(source=target[0])

    # return 0 or None upon success
    return None

# Tangle to .cc and .o
builder = Builder(
    action=build_function,
    suffix='.cc',
    src_suffix='.nw',
    emitter=cpp_emit)

env = Environment()
env['BUILDERS']['tangle'] = builder

tangleTarget = env.tangle(target='main.cc', source='main.nw')

Here is the output:

$ scons
scons: Reading SConscript files ...
Emitter source: main.nw
Emitter target: main.cc
scons: done reading SConscript files.
scons: Building targets ...
build_function(["main.cc"], ["main.nw"])
Builder target: main.cc
Builder source: main.nw
scons: done building targets.

$ scons -c
scons: Reading SConscript files ...
Emitter source: main.nw
Emitter target: main.cc
scons: done reading SConscript files.
scons: Cleaning targets ...
Removed main.cc
scons: done cleaning targets.

In this second example, if you add the object file as a target, you will get the following error (rightly so)

scons: *** [main.cc] Multiple ways to build the same target were specified for: main.o  (from ['main.nw'] and from ['main.cc'])