Advanced Deep Learning with Keras

The add operation can be implemented in Keras by the add() merge function.
However, both the F ( x
l−1
) equation and x should have the same dimensions.
If the dimensions are different, for example, when changing the feature maps
size, we should perform a linear projection on x as to match the size of F ( x
l−1
) .
In the original paper, the linear projection for the case, when the feature maps
size is halved, is done by a Conv2D with a 1 × 1 kernel and strides=2.
Chapter 2
Back in Chapter 1, Introducing Advanced Deep Learning with Keras, we discussed
that stride > 1 is equivalent to skipping pixels during convolution. For example,
if strides=2, we could skip every other pixel when we slide the kernel during the
convolution process.
The preceding Equations 2.2.3 and 2.2.4, both model ResNet residual block
operations. They imply that if the deeper layers can be trained to have fewer
errors, then there is no reason why the shallower layers should have higher errors.
Knowing the basic building blocks of ResNet, we're able to design a deep residual
network for image classification. This time, however, we're going to tackle a more
challenging and advanced dataset.
In our examples, we're going to consider CIFAR10, which was one of the datasets the
original paper was validated. In this example, Keras provides an API to conveniently
access the CIFAR10 dataset, as shown:
from keras.datasets import cifar10
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
Like MNIST, the CIFAR10 dataset has 10 categories. The dataset is a collection of
small (32 × 32) RGB real-world images of an airplane, automobile, bird, cat, deer,
dog, frog, horse, ship, and a truck corresponding to each of the 10 categories. Figure
2.2.4 shows sample images from CIFAR10.
[ 53 ]