Where is the Backward function defined in PyTorch?

Question

This might sound a little basic but while running the code below, I wanted to see the source code of the backward function:

import torch.nn as nn

[...]
criterion = nn.CrossEntropyLoss()
loss = criterion(output, target)
loss.backward()

So I went to the PyTorch GitHub and found the CrossEntropyLoss class, but without any backward function defined. Moving up, CrossEntropyLoss extends _WeightedLoss >> _Loss >> Module then still nothing.

So, where is the backward function defined?

Answer 1

Backward function is same for all type of layers.

Look at 155 number line here.

https://github.com/pytorch/pytorch/blob/35bd2b3c8b64d594d85fc740e94c30aa67892a34/torch/tensor.py

and it will forward you to here

https://github.com/pytorch/pytorch/blob/35bd2b3c8b64d594d85fc740e94c30aa67892a34/torch/autograd/__init__.py