batch_size in neural network

Question

When NN is construsted, batch size is not defined and place holder is used and its summary(tensorfow) shows the batch size as None.

This is useful because you can change batch size later.

In case of a simple model with 10 input features, 1 hidden layers with 10 neurons and output layer, the shape of the hidden layer would be (None, 10), which means if the batch size is 20, the hidden layer would have the shape of (20, 10).

When the model is used to predict for a single output, with shape(10, 1), how does the math work?

Answer 1

I guess you have a confusion here. The None part represents the number of samples.

For example if you have a neural network with architecture 100-50-10, it means that you have

• (None,100) : input layer shape
• (100,50) : shape for weights connecting input to hidden layer
• (None,50): shape for hidden layer given by (None,100)*(100,50) matrix multiplication
• (None,50): shape after nonlinearity application.
• (50,10): shape for the shape matrix between hidden and output layer
• (None,10) : output layer shape (None,50)*(50,10) matrix multiplication

So if youre feeding a single input sample the shapes would be:

(1,100)[Input] => (1,100)(100,50) = (1,50)[Hidden Layer] => (1,50)*(50,10)=(1,10)[Output Layer]