Advanced Deep Learning with Keras

Introducing Advanced Deep Learning with Keras
Since optimization is based on differentiation, it follows that an important criterion
of the loss function is that it must be smooth or differentiable. This is an important
constraint to keep in mind when introducing a new loss function.
Given the training dataset, the choice of the loss function, the optimizer, and the
regularizer, the model can now be trained by calling the fit() function:
# loss function for one-hot vector
# use of adam optimizer
# accuracy is a good metric for classification tasks
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
# train the network
model.fit(x_train, y_train, epochs=20, batch_size=batch_size)
This is another helpful feature of Keras. By just supplying both the x and y data,
the number of epochs to train, and the batch size, fit() does the rest. In other deep
learning frameworks, this translates to multiple tasks such as preparing the input
and output data in the proper format, loading, monitoring, and so on. While all of
these must be done inside a for loop! In Keras, everything is done in just one line.
In the fit() function, an epoch is the complete sampling of the entire training data.
The batch_size parameter is the sample size of the number of inputs to process at
each training step. To complete one epoch, fit() requires the size of train dataset
divided by batch size, plus 1 to compensate for any fractional part.
Performance evaluation
At this point, the model for the MNIST digit classifier is now complete. Performance
evaluation will be the next crucial step to determine if the proposed model has come
up with a satisfactory solution. Training the model for 20 epochs will be sufficient to
obtain comparable performance metrics.
The following table, Table 1.3.2, shows the different network configurations and
corresponding performance measures. Under Layers, the number of units is shown
for layers 1 to 3. For each optimizer, the default parameters in Keras are used. The
effects of varying the regularizer, optimizer and number of units per layer can be
observed. Another important observation in Table 1.3.2 is that bigger networks do
not necessarily translate to better performance.
Increasing the depth of this network shows no added benefits in terms of accuracy for
both training and testing datasets. On the other hand, a smaller number of units, like
128, could also lower both the test and train accuracy. The best train accuracy at 99.93%
is obtained when the regularizer is removed, and 256 units per layer are used. The test
accuracy, however, is much lower at 98.0%, as a result of the network overfitting.
[ 20 ]