PyBrain multiple target values

Question

I'm trying to train ANN to predict probabilities of an image belonging to a several number of classes, and my target values are sets of such probabilities.

Input is simple reshaped 28x28 grayscale pictures with pixel values from 0-255.

One 'target' looks like this: 0.738832,0.238159,0.023009,0,0.238159,0,0.238159,0,0.238159,0,0,0.238159,0,0.19793,0.80207,0.066806667,0.663691308,0.008334764,0,0,0.0494825,0.098965,0.0494825,0,0,0,0,0,0,0,0,0,0,0,0,0,0

However the results that I get are terrible (far worse than simple linear regression) and looks like this: 0.011947,0.448668,0,0,0.095688,0,0.038233,0,0,0,0,0,0,0,0.405464,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

It doesn't really matter if I use 300 or 30000 pictures. I'm obviously doing something wrong and I really appreciate some advice.

Code:

# create dataset
DS = SupervisedDataSet(784, 37)
assert(ia.shape[0] == ta.shape[0])
DS.setField('input', ia)
DS.setField('target', ta)

fnn = buildNetwork( DS.indim, 200, 37, outclass=SoftmaxLayer )

trainer = BackpropTrainer( fnn, dataset=DS, momentum=0.1, verbose=True, weightdecay=0.01)
trainer.trainUntilConvergence(maxEpochs=10,verbose=True,validationProportion=0.20)

Answer 1

Your problem is the values you are using for training. A softmax layer means that all values for that layer will sum to one. So when you setup 37 output dimensions that means all 37 dimensions will sum to 1.0. Your sample target does not appear to follow that distribution.