pdfcoffee
Chapter 1[ 0., 1., 0.],[ 0., 0., 1.],[ 1., 0., 0.]], dtype=float32)Let's run the code and see what results we get with this multi-layer network:Figure 15: Running the code for a multi-layer networkThe previous screenshot shows the initial steps of the run while the followingscreenshot shows the conclusion. Not bad. As seen in the following screenshot,by adding two hidden layers we reached 90.81% on the training set, 91.40% onvalidation, and 91.18% on test. This means that we have increased accuracy ontesting with respect to the previous network, and we have reduced the numberof iterations from 200 to 50. That's good, but we want more.If you want, you can play by yourself and see what happens if you add onlyone hidden layer instead of two or if you add more than two layers. I leave thisexperiment as an exercise:Figure 16: Results after adding two hidden layers, with accuracies shownNote that improvement stops (or they become almost imperceptible) after a certainnumber of epochs. In machine learning, this is a phenomenon called convergence.[ 23 ]
Neural Network Foundations with TensorFlow 2.0Further improving the simple netin TensorFlow with DropoutNow our baseline is 90.81% on the training set, 91.40% on validation, and 91.18%on test. A second improvement is very simple. We decide to randomly drop – withthe DROPOUT probability – some of the values propagated inside our internal densenetwork of hidden layers during training. In machine learning this is a well-knownform of regularization. Surprisingly enough, this idea of randomly dropping a fewvalues can improve our performance. The idea behind this improvement is thatrandom dropout forces the network to learn redundant patterns that are usefulfor better generalization:import tensorflow as tfimport numpy as npfrom tensorflow import keras# Network and training.EPOCHS = 200BATCH_SIZE = 128VERBOSE = 1NB_CLASSES = 10 # number of outputs = number of digitsN_HIDDEN = 128VALIDATION_SPLIT = 0.2 # how much TRAIN is reserved for VALIDATIONDROPOUT = 0.3# Loading MNIST dataset.# Labels have one-hot representation.mnist = keras.datasets.mnist(X_train, Y_train), (X_test, Y_test) = mnist.load_data()# X_train is 60000 rows of 28x28 values; we reshape it to 60000 x 784.RESHAPED = 784#X_train = X_train.reshape(60000, RESHAPED)X_test = X_test.reshape(10000, RESHAPED)X_train = X_train.astype('float32')X_test = X_test.astype('float32')# Normalize inputs within [0, 1].X_train, X_test = X_train / 255.0, X_test / 255.0print(X_train.shape[0], 'train samples')print(X_test.shape[0], 'test samples')# One-hot representations for labels.[ 24 ]
- Page 8 and 9: Sujit Pal is a Technology Research
- Page 10 and 11: Table of ContentsPrefacexiChapter 1
- Page 12 and 13: [ iii ]Table of ContentsConverting
- Page 14 and 15: Table of ContentsSo what is the pro
- Page 16 and 17: [ vii ]Table of ContentsChapter 10:
- Page 18 and 19: Table of ContentsPretrained models
- Page 20 and 21: PrefaceDeep Learning with TensorFlo
- Page 22 and 23: • Supervised learning, in which t
- Page 24 and 25: PrefaceThe complexity of deep learn
- Page 26 and 27: PrefaceFigure 5: Adoption of deep l
- Page 28 and 29: Chapter 1, Neural Network Foundatio
- Page 30 and 31: PrefaceChapter 13, TensorFlow for M
- Page 32 and 33: ConventionsThere are a number of te
- Page 34: PrefaceReferences1. Deep Learning w
- Page 37 and 38: Neural Network Foundations with Ten
- Page 39 and 40: Neural Network Foundations with Ten
- Page 41 and 42: Neural Network Foundations with Ten
- Page 43 and 44: Neural Network Foundations with Ten
- Page 45 and 46: Neural Network Foundations with Ten
- Page 47 and 48: Neural Network Foundations with Ten
- Page 49 and 50: Neural Network Foundations with Ten
- Page 51 and 52: Neural Network Foundations with Ten
- Page 53 and 54: Neural Network Foundations with Ten
- Page 55 and 56: Neural Network Foundations with Ten
- Page 57: Neural Network Foundations with Ten
- Page 61 and 62: Neural Network Foundations with Ten
- Page 63 and 64: Neural Network Foundations with Ten
- Page 65 and 66: Neural Network Foundations with Ten
- Page 67 and 68: Neural Network Foundations with Ten
- Page 69 and 70: Neural Network Foundations with Ten
- Page 71 and 72: Neural Network Foundations with Ten
- Page 73 and 74: Neural Network Foundations with Ten
- Page 75 and 76: Neural Network Foundations with Ten
- Page 77 and 78: Neural Network Foundations with Ten
- Page 79 and 80: Neural Network Foundations with Ten
- Page 81 and 82: Neural Network Foundations with Ten
- Page 83 and 84: Neural Network Foundations with Ten
- Page 86 and 87: TensorFlow 1.x and 2.xThe intent of
- Page 88 and 89: An example to start withWe'll consi
- Page 90 and 91: Chapter 23. Placeholders: Placehold
- Page 92 and 93: • To create random values from a
- Page 94 and 95: To know the value, we need to creat
- Page 96 and 97: Chapter 2Both PyTorch and TensorFlo
- Page 98 and 99: Chapter 2state = [tf.zeros([100, 10
- Page 100 and 101: Chapter 2For now, there's no need t
- Page 102 and 103: Chapter 2Let's see an example of a
- Page 104 and 105: Chapter 2If you want to save a mode
- Page 106 and 107: Chapter 2supervised=True)train_data
Chapter 1
[ 0., 1., 0.],
[ 0., 0., 1.],
[ 1., 0., 0.]], dtype=float32)
Let's run the code and see what results we get with this multi-layer network:
Figure 15: Running the code for a multi-layer network
The previous screenshot shows the initial steps of the run while the following
screenshot shows the conclusion. Not bad. As seen in the following screenshot,
by adding two hidden layers we reached 90.81% on the training set, 91.40% on
validation, and 91.18% on test. This means that we have increased accuracy on
testing with respect to the previous network, and we have reduced the number
of iterations from 200 to 50. That's good, but we want more.
If you want, you can play by yourself and see what happens if you add only
one hidden layer instead of two or if you add more than two layers. I leave this
experiment as an exercise:
Figure 16: Results after adding two hidden layers, with accuracies shown
Note that improvement stops (or they become almost imperceptible) after a certain
number of epochs. In machine learning, this is a phenomenon called convergence.
[ 23 ]
Change language