pdfcoffee

Neural Network Foundations with TensorFlow 2.0
Further improving the simple net
in TensorFlow with Dropout
Now our baseline is 90.81% on the training set, 91.40% on validation, and 91.18%
on test. A second improvement is very simple. We decide to randomly drop – with
the DROPOUT probability – some of the values propagated inside our internal dense
network of hidden layers during training. In machine learning this is a well-known
form of regularization. Surprisingly enough, this idea of randomly dropping a few
values can improve our performance. The idea behind this improvement is that
random dropout forces the network to learn redundant patterns that are useful
for better generalization:
import tensorflow as tf
import numpy as np
from tensorflow import keras
# Network and training.
EPOCHS = 200
BATCH_SIZE = 128
VERBOSE = 1
NB_CLASSES = 10 # number of outputs = number of digits
N_HIDDEN = 128
VALIDATION_SPLIT = 0.2 # how much TRAIN is reserved for VALIDATION
DROPOUT = 0.3
# Loading MNIST dataset.
# Labels have one-hot representation.
mnist = keras.datasets.mnist
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
# X_train is 60000 rows of 28x28 values; we reshape it to 60000 x 784.
RESHAPED = 784
#
X_train = X_train.reshape(60000, RESHAPED)
X_test = X_test.reshape(10000, RESHAPED)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
# Normalize inputs within [0, 1].
X_train, X_test = X_train / 255.0, X_test / 255.0
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
# One-hot representations for labels.
[ 24 ]