What is identity mapping in neural networks

Question

I came across the term identity mapping in some papers about neural networks but am not sure what it is supposed to mean in that context. I'm guessing it means to map a sort of input to an output?

I came across this term in the papers Deep Residual Learning for Image Recognition (2015) and Identity Mappings in Deep Residual Networks (2016), both by He et al.

Answer 1

I'm not an neural network expert but I understand that identity mapping ensures that the output of some multilayer neural net is ensured to be equal to its input. Such a net is also called a replicator.

I have understood that such identity/replication facilitates unsupervised training, and that the hidden layers of such nets can be used for feature detection and data compression.

Further reading:

• neural networks and non linear principal compinent analysis with a nice example of auto associative network.
• difference between replicator networks and autoencoders
• Approximate realization of identity mappings by three‐layer neural networks abstract of an article that summarizes properties of the hidden layer

Answer 2

It's the same as it is in Algebra.

An identity map or identity function gives out exactly what it got.

When they say:

h(x_l) = x_l

They mean h is an identity mapping / function. If you give it x_l it will give you back x_l.

h might be something else but once they say it's h(x_l) = x_l then it's an identity map / function.

I don't see anything here to suggest the meaning changes once you're in a neural network context.

It might seem pointless to have a map or function that gives you back what you gave it, and it kind of is. The need for them arises from using a system (an architecture, framework, or whatever) that expects a map/function in a step that actually isn't needed in this case. Rather than build a new system for this case we use an identity map/function that lets us reuse the old system.

It's the same kind of thinking that invented 0, the null object pattern, the option type, and some of the reasons I pass in empty collections. The system expects something but you need that something to be nothing. These constructs give the nothing a form that wont break the system.