Do I need to square a column if I want a neural network to try using that?

Question

In the classical linear regression implementation, if I suspect the square of the values of the column is correlated to the target, then I actually need to create a new column with the squares for the algorithm to make use of that.

Is this also necessary when using neural networks? I know it's a broad question - are there cases where this is necessary and cases where it isn't?

Answer 1

You don’t necessarily need to, according to the universal function approximation theorem.

• It is easier for a neural network to learn an identity function than some other function, so if one of the inputs definitely needs to be squared your network will learn faster if you pass the input already squared
• If your network is sufficiently large it should work out that squaring that input is helpful and approximate the squaring function as part of the overall learning process