Implementing the SVHN CNN architecture in Srivastava et al. 2014 Dropout paper

Question

I am trying the implement the CNN architecture introduced in Srivastava et al. 2014 Dropout paper (appendix B.2), for the SVHN dataset. I implemented only the convolutional layers part, without dropout or any regularization for the moment, following this description of the mentioned paper: "The convolutional layers have 96, 128 and 256 filters respectively. Each convolutional layer has a 5 × 5 receptive field applied with a stride of 1 pixel. Each max pooling layer pools 3 × 3 regions at strides of 2 pixels."

This is my code, implemented in TensorFlow 2.0 with Keras API

    from tensorflow.keras import layers, Sequential

    model = Sequential(name= "fMap_svhn_DANN")

    model.add(layers.Conv2D(filters= 96, kernel_size= 5, activation= 'relu', input_shape= (32,32,3)))
    model.add(layers.MaxPool2D(pool_size= 3, strides= 2))

    model.add(layers.Conv2D(filters=128, kernel_size= 5, activation= 'relu'))
    model.add(layers.MaxPool2D(pool_size= 3, strides= 2))

    model.add(layers.Conv2D(filters= 256, kernel_size= 5, activation= 'relu'))
    model.add(layers.MaxPool2D(pool_size= 3, strides= 2))

    model.add(layers.Flatten())

And this is the error I get"

Negative dimension size caused by subtracting 5 from 4 for 'conv2d_11/Conv2D' (op: 'Conv2D') with input shapes: [?,4,4,128], [5,5,128,256].

Any idea to help me ?

Answer 1

This means that your input shape was too small by the time it reached the last Conv2D layer.

By removing the last Conv2D and MaxPool2D layer, this is how the model looks like:

You can see that the last output before the Flatten layer has height and width of 4, which is smaller than your kernel_size of 5 in the last Conv2D layer.

Heres what you can do:

1. Increase the input image height and width OR
2. Set padding='same' in the Conv2D layers