Advanced Deep Learning with Keras

Deep Neural Networks
Generally speaking, we can observe that the layer l output feature maps are directly
affected by the previous feature maps only. Meanwhile, for ResNet:
x
( x )
l-1 = H
(Equation 2.2.3)
l-2
( ( -1 ) -2 )
x = ReLU F x + x
(Equation 2.2.4)
l l l
F
( )
Figure 2.2.3: A detailed layer operations for a plain CNN block and a Residual block
x
l−1
is made of Conv2D-BN, which is also known as the residual mapping.
The + sign is tensor element-wise addition between the shortcut connection and
the output of F ( x
l−1
) . The shortcut connection doesn't add extra parameters nor
extra computational complexity.
[ 52 ]