pdfcoffee
ConvolutionalNeural NetworksIn the previous chapters we have discussed DenseNets, in which each layer isfully connected to the adjacent layers. We looked at one application of these densenetworks in classifying the MNIST handwritten characters dataset. In that context,each pixel in the input image has been assigned to a neuron with a total of 784(28 × 28 pixels) input neurons. However, this strategy does not leverage the spatialstructure and relationships between each image. In particular, this piece of codeis a DenseNet that transforms the bitmap representing each written digit into a flatvector where the local spatial structure is removed. Removing the spatial structureis a problem because important information is lost:#X_train is 60000 rows of 28x28 values --> reshaped in 60000 x 784X_train = X_train.reshape(60000, 784)X_test = X_test.reshape(10000, 784)Convolutional neural networks (in short, convnets or CNNs) leverage spatialinformation, and they are therefore very well-suited for classifying images. Thesenets use an ad hoc architecture inspired by biological data taken from physiologicalexperiments performed on the visual cortex. As we discussed in Chapter 2, TensorFlow1.x and 2.x, our vision is based on multiple cortex levels, each one recognizing moreand more structured information. First, we see single pixels, then from that werecognize simple geometric forms and then more and more sophisticated elementssuch as objects, faces, human bodies, animals, and so on.Convolutional neural networks are a fascinating subject. Over a short period of time,they have shown themselves to be a disruptive technology, breaking performancerecords in multiple domains from text, to video, to speech, going well beyond theinitial image processing domain where they were originally conceived.[ 109 ]
Convolutional Neural NetworksIn this chapter we will introduce the idea of CNNs a particular type of neuralnetworks that have large importance for deep learning.Deep Convolutional Neural Network(DCNN)A Deep Convolutional Neural Network (DCNN) consists of many neuralnetwork layers. Two different types of layers, convolutional and pooling (that is,subsampling), are typically alternated. The depth of each filter increases from left toright in the network. The last stage is typically made of one or more fully connectedlayers:Figure 1: An example of a DCNNThere are three key underlying concepts for convnets: local receptive fields, sharedweights, and pooling. Let's review them together.Local receptive fieldsIf we want to preserve the spatial information of an image or other form of data, thenit is convenient to represent each image with a matrix of pixels. Given this, a simpleway to encode the local structure is to connect a submatrix of adjacent input neuronsinto one single hidden neuron belonging to the next layer. That single hidden neuronrepresents one local receptive field. Note that this operation is named convolution,and this is where the name for this type of network is derived. You can think aboutconvolution as the treatment of a matrix by another matrix, referred to as a kernel.Of course, we can encode more information by having overlapping submatrices. Forinstance, let's suppose that the size of each single submatrix is 5×5 and that thosesubmatrices are used with MNIST images of 28×28 pixels, then we will be able togenerate 24×24 local receptive field neurons in the hidden layer.[ 110 ]
- Page 94 and 95: To know the value, we need to creat
- Page 96 and 97: Chapter 2Both PyTorch and TensorFlo
- Page 98 and 99: Chapter 2state = [tf.zeros([100, 10
- Page 100 and 101: Chapter 2For now, there's no need t
- Page 102 and 103: Chapter 2Let's see an example of a
- Page 104 and 105: Chapter 2If you want to save a mode
- Page 106 and 107: Chapter 2supervised=True)train_data
- Page 108 and 109: Chapter 2There, tf.feature_column.n
- Page 110 and 111: Chapter 2print (dz_dx)print (dy_dx)
- Page 112 and 113: Chapter 2In our toy example we use
- Page 114 and 115: Chapter 2For multi-machine training
- Page 116 and 117: Chapter 25. Use tf.layers modules t
- Page 118 and 119: Chapter 2Keras or tf.keras?Another
- Page 120: • tf.data can be used to load mod
- Page 123 and 124: RegressionLet us imagine a simpler
- Page 125 and 126: RegressionTake a look at the last t
- Page 127 and 128: Regression3. Now, we calculate the
- Page 129 and 130: RegressionIn the next section we wi
- Page 131 and 132: Regression2. Now, we define the fea
- Page 133 and 134: Regression2. Download the dataset:(
- Page 135 and 136: RegressionThe following is the Tens
- Page 137 and 138: RegressionIn regression the aim is
- Page 139 and 140: RegressionThe Estimator outputs the
- Page 141 and 142: RegressionThe following is the grap
- Page 143: RegressionReferencesHere are some g
- Page 147 and 148: Convolutional Neural NetworksIn thi
- Page 149 and 150: Convolutional Neural NetworksIn oth
- Page 151 and 152: Convolutional Neural NetworksThen w
- Page 153 and 154: Convolutional Neural NetworksHoweve
- Page 155 and 156: Convolutional Neural NetworksPlotti
- Page 157 and 158: Convolutional Neural NetworksIn gen
- Page 159 and 160: Convolutional Neural NetworksOur ne
- Page 161 and 162: Convolutional Neural NetworksThese
- Page 163 and 164: Convolutional Neural NetworksSo, we
- Page 165 and 166: Convolutional Neural NetworksEach i
- Page 167 and 168: Convolutional Neural NetworksVery d
- Page 169 and 170: Convolutional Neural NetworksRecogn
- Page 171 and 172: Convolutional Neural NetworksIf we
- Page 173 and 174: Convolutional Neural NetworksRefere
- Page 175 and 176: Advanced Convolutional Neural Netwo
- Page 177 and 178: Advanced Convolutional Neural Netwo
- Page 179 and 180: Advanced Convolutional Neural Netwo
- Page 181 and 182: Advanced Convolutional Neural Netwo
- Page 183 and 184: Advanced Convolutional Neural Netwo
- Page 185 and 186: Advanced Convolutional Neural Netwo
- Page 187 and 188: Advanced Convolutional Neural Netwo
- Page 189 and 190: Advanced Convolutional Neural Netwo
- Page 191 and 192: Advanced Convolutional Neural Netwo
- Page 193 and 194: Advanced Convolutional Neural Netwo
Convolutional Neural Networks
In this chapter we will introduce the idea of CNNs a particular type of neural
networks that have large importance for deep learning.
Deep Convolutional Neural Network
(DCNN)
A Deep Convolutional Neural Network (DCNN) consists of many neural
network layers. Two different types of layers, convolutional and pooling (that is,
subsampling), are typically alternated. The depth of each filter increases from left to
right in the network. The last stage is typically made of one or more fully connected
layers:
Figure 1: An example of a DCNN
There are three key underlying concepts for convnets: local receptive fields, shared
weights, and pooling. Let's review them together.
Local receptive fields
If we want to preserve the spatial information of an image or other form of data, then
it is convenient to represent each image with a matrix of pixels. Given this, a simple
way to encode the local structure is to connect a submatrix of adjacent input neurons
into one single hidden neuron belonging to the next layer. That single hidden neuron
represents one local receptive field. Note that this operation is named convolution,
and this is where the name for this type of network is derived. You can think about
convolution as the treatment of a matrix by another matrix, referred to as a kernel.
Of course, we can encode more information by having overlapping submatrices. For
instance, let's suppose that the size of each single submatrix is 5×5 and that those
submatrices are used with MNIST images of 28×28 pixels, then we will be able to
generate 24×24 local receptive field neurons in the hidden layer.
[ 110 ]
Change language