pdfcoffee

Chapter 8
Perhaps the best approach is to replace the current loss function with one that ignores
matches where both numbers are zero; however, a simpler approach is to build
a stricter metric and use that to judge when to stop the training. Accordingly, we
build a new accuracy function masked_accuracy() whose code is shown as follows:
def masked_accuracy():
def masked_accuracy_fn(ytrue, ypred):
ytrue = tf.keras.backend.argmax(ytrue, axis=-1)
ypred = tf.keras.backend.argmax(ypred, axis=-1)
mask = tf.keras.backend.cast(
tf.keras.backend.not_equal(ypred, 0), tf.int32)
matches = tf.keras.backend.cast(
tf.keras.backend.equal(ytrue, ypred), tf.int32) * mask
numer = tf.keras.backend.sum(matches)
denom = tf.keras.backend.maximum(tf.keras.backend.sum(mask), 1)
accuracy = numer / denom
return accuracy
return masked_accuracy_fn
We are now ready to train our model. As usual, we set up the model checkpoint and
TensorBoard callbacks, and then call the fit() convenience method on the model
to train the model with a batch size of 128 for 50 epochs:
num_epochs = 50
best_model_file = os.path.join(data_dir, "best_model.h5")
checkpoint = tf.keras.callbacks.ModelCheckpoint(
best_model_file,
save_weights_only=True,
save_best_only=True)
tensorboard = tf.keras.callbacks.TensorBoard(log_dir=logs_dir)
history = model.fit(train_dataset,
epochs=num_epochs,
validation_data=val_dataset,
callbacks=[checkpoint, tensorboard])
A truncated output of the training is shown as follows. As you can see, the masked_
accuracy and val_masked_accuracy numbers seem more conservative than the
accuracy and val_accuracy numbers. This is because the masked versions do not
consider the sequence positions where the input is a PAD character:
Epoch 1/50
19/19 [==============================] - 8s 431ms/step - loss: 1.4363 -
accuracy: 0.7511 - masked_accuracy_fn: 0.00
[ 313 ]