Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

It should be pretty clear, except for one small detail: It may be necessary to
downsample the input.
"Why is that?"
To add up two images, they must have the same dimensions—not only the height
and the width, but also the number of channels (adding up is not the same as
stacking up channels!). That poses a problem for the residual block, since the
number of output channels of the last convolutional layer may be different than
the number of channels in the input.
If only there were an operation that took the original input and generated an
output with a different number of channels—do you know any?
"What about a convolutional layer?"
Bingo! We can use yet another convolutional layer to produce an input (now
modified) that has a matching number of channels so it can be added to the main
output.
"But then it is not the original input anymore, is it?"
Not really, no, because it will be modified by the downsampling convolutional layer.
But, even though it goes somewhat against the idea of learning the identity
function, the usefulness of a shortcut still stands.
Residual Connections | 553