How is it possible to process an image with a few neurons?

Question

An 1024*1024 pixel image has around one million pixels. If I would like to connect each pixel to an R,G,B input neuron, then more than 3 million neurons are needed.

It would be really hard, to train a neural network, which has millions of inputs. How is it possible, to reduce the number of neurons?

Answer 1

There are several ways to make this big number trainable:

• Use CNNs
• Auto-Encoders (see Reducing the Dimensionality of Data with Neural Networks)

Dimensionality reduction of the input

• Scale the image down
• PCA / LDA

Troll-Answer

If you really meant "only a few neurons" then you might want to have a look at Spiking neural networks. Those are incredibly computationally intensive, need a lot of hand-crafting and still get worse performance than normal neural networks for most tasks ... but you only need very little of them.

Answer 2

Convolutional neural nets share weights across the whole input image and this drastically reduces the number of weights. For example you could have a 3x3 grid where each point has its own weight, this scans over each of the 3 channels in the 1024x1024 image. Only 9 weights are used for each channel making 27 weights. If you have, say 10 of these grids then that's only 270 weights!

Furthermore by sharing weights you create some translation invariance over the input space which is a desirable property in things like object recognition.