Error when formatting large string - How to detect error?

Question

I am pretty new to Python and I am currently working on large string formatting that I need for a library I am using.

The problem occurs as I do not understand where exactly the error is happening within the large string format. More precisely I get an error of the form

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2731, in run_code
    exec code_obj in self.user_global_ns, self.user_ns
  File "<ipython-input-3-f6a3bb7fe2f9>", line 13, in <module>
    trainCV = trainCV % (train_params)
ValueError: unsupported format character ',' (0x2c) at index 2726 

Is there an way to precisely detect the line get where the error occurs ?

My complete code looks like this:

trainCV = open('Conv_Constructor.yaml','r').read()
train_params    =   {'batch_size': 100,
                     'output_channels_h2': 64,
                     'conv_kernel_size': 8,
                     'pool_size': 2,
                     'stride_size': 1,
                     'output_channels_h3': 64,
                     'num_classes': 6,
                     'valid_stop': 4200,
                     'test_start': 4200,
                     'test_stop': 4400,
                     'max_epochs': 5}

trainCV = trainCV % (train_params)
print trainCV

And the Conv_Constructor.yaml file I am trying to format as a string is the following

#  ---------- INPUTS ---------
#
#        batch_size
#        output_channels_h2
#        conv_kernel_size
#        pool_size
#        stride_size
#        output_channels_h3
#        num_classes
#        valid_stop
#        test_start
#        test_stop
#        max_epochs

##################################################################

!obj:pylearn2.train.Train {
    dataset: !obj:pylearn2.official_train_data.load_data {
        start: 0,
        stop:  4000
      # one_hot: 1,
    },

    model: !obj:pylearn2.models.mlp.MLP {
        batch_size: %(batch_size)i,
        input_space: !obj:pylearn2.space.Conv2DSpace {
            shape: [32, 32],
            num_channels: 1,
            axes = ('b',0,1,'c')
        },

        layers: [ !obj:pylearn2.models.mlp.ConvRectifiedLinear {
                     layer_name: 'h2',
                     output_channels: %(output_channels_h2)i,
                     #params : !pkl: 'dae_layer_1_weights.plk',
                     irange: .05,

                     kernel_shape: [%(conv_kernel_size)i, %(conv_kernel_size)i],

                     pool_shape: [%(pool_size)i, %(pool_size)i],

                     pool_stride: [%(stride_size)i, %(stride_size)i],

                     max_kernel_norm: 1.9365
                 }, !obj:pylearn2.models.mlp.ConvRectifiedLinear {
                     layer_name: 'h3',
                     output_channels: %(output_channels_h3)i,
                     #params : !pkl: 'dae_layer_1_weights.plk',
                     irange: .05,
                     kernel_shape: %(conv_kernel_size)i, %(conv_kernel_size)i],

                     pool_shape:[%(pool_size)i, %(pool_size)i],

                     pool_stride: [%(stride_size)i, %(stride_size)i],
                     max_kernel_norm: 1.9365


                 },  !obj:pylearn2.models.mlp.Softmax {
                     max_col_norm: 1.9365,
                     layer_name: 'y',
                     n_classes: %(num_classes)i,
                     istdev: .05
                 }
                ],
    },

    algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
        batch_size: %(batch_size)i,
        learning_rate: .01,
        init_momentum: .5,
        monitoring_dataset:
            {
                'valid' : !obj:pylearn2.official_train_data.load_data {
                         start: 4000,
                         stop:  %(valid_stop)i
                         #one_hot: 1,
                },
                'test'  :  !obj:pylearn2.official_train_data.load_data {
                         start: %(test_start),
                         stop:  %(test_stop)
                         #one_hot: 1,
                }
            },

        cost: !obj:pylearn2.costs.cost.SumOfCosts { costs: [
            !obj:pylearn2.costs.cost.MethodCost {
                method: 'cost_from_X'
            }, !obj:pylearn2.costs.mlp.WeightDecay {
                coeffs: [ .00005, .00005, .00005 ]
            }
            ]
        },
        termination_criterion: !obj:pylearn2.termination_criteria.And {
            criteria: [
                !obj:pylearn2.termination_criteria.MonitorBased {
                    channel_name: "valid_y_misclass",
                    prop_decrease: 0.50,
                    N: 50
                },
                !obj:pylearn2.termination_criteria.EpochCounter {
                    max_epochs: %(max_epochs)i
                },
            ]
        },
    },

    extensions:
        [ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
             channel_name: 'valid_y_misclass',
             save_path: "%(save_path)s/convolutional_network_best.pkl"
        }, !obj:pylearn2.training_algorithms.sgd.MomentumAdjustor {
            start: 1,
            saturate: 10,
            final_momentum: .99
        }
    ]
}

Answer 1

You can locate the error more easily by processing each line separately instead of your whole string. Replace

trainCV = trainCV % (train_params)

with

trainCV = trainCV.split('\n')
t1=[]
try:
    for i, t in enumerate(trainCV):
        t1.append(t % (train_params))
except :
    print 'Error in line {}:'.format(i)
    print t[i]
    raise

and you will get the following output:

78
                         start: %(test_start),

meaning your string formating didn't quite work (in this case I think there is the i after the bracket missing). Debug your large string in this way and you should have a working code.

After that is done you can print it by joining the list:

print '\n'.join(t1)