pdfcoffee
Chapter 10In SOMs, neurons are usually placed at nodes of a (1D or 2D) lattice. Higherdimensions are also possible but are rarely used in practice. Each neuron in thelattice is connected to all the input units via a weight matrix. The following diagramshows a SOM with 6 × 8 (48 neurons) and 5 inputs. For clarity, only the weightvectors connecting all inputs to one neuron are shown. In this case, each neuron willhave seven elements, resulting in a combined weight matrix of size (40 × 5):A SOM learns via competitive learning. It can be considered as a nonlineargeneralization of PCA and thus, like PCA, can be employed for dimensionalityreduction.In order to implement SOM, let's first understand how it works. As a first step,the weights of the network are initialized either to some random value or by takingrandom samples from the input. Each neuron occupying a space in the lattice will beassigned specific locations. Now as an input is presented, the neuron with the leastdistance from the input is declared the winner (WTU). This is done by measuring thedistance between the weight vectors (W) and input vectors (X) of all neurons:NNdd jj = √∑(WW jjjj − XX ii ) 2ii=1Here, d jis the distance of weights of neuron j from input X. The neuron with thelowest d value is the winner.Next, the weights of the winning neuron and its neighboring neurons are adjusted ina manner to ensure that the same neuron is the winner if the same input is presentednext time.[ 385 ]
Unsupervised LearningTo decide which neighboring neurons need to be modified, the network usesa neighborhood function ∧ (rr) ; normally, the Gaussian Mexican hat function ischosen as a neighborhood function. The neighborhood function is mathematicallyrepresented as follows:∧ (rr) = ee − dd22σσ 2Here, σσ is a time-dependent radius of influence of a neuron and d is its distance fromthe winning neuron. Graphically the function looks like a hat (hence its name), asyou can see in the following figure:Figure 6: The "Gaussian Maxican hat" function, visualized in graph formAnother important property of the neighborhood function is that its radius reduceswith time. As a result, in the beginning, many neighboring neurons' weights aremodified, but as the network learns, eventually a few neurons' weights (at times,only one or none) are modified in the learning process. The change in weight is givenby the following equation:dddd = ηη ∧ (XX − WW)The process is repeated for all the inputs for a given number of iterations. As theiterations progress, we reduce the learning rate and the radius by a factor dependenton the iteration number.SOMs are computationally expensive and thus are not really useful for verylarge datasets. Still, they are easy to understand, and they can very nicely findthe similarity between input data. Thus, they have been employed for imagesegmentation and to determine word similarity maps in NLP [3].[ 386 ]
- Page 369 and 370: Recurrent Neural Networksself.atten
- Page 371 and 372: Recurrent Neural Networks30 try to
- Page 373 and 374: Recurrent Neural Networks3. Because
- Page 375 and 376: Recurrent Neural NetworksSummaryIn
- Page 377 and 378: Recurrent Neural Networks18. Shi, X
- Page 380 and 381: AutoencodersAutoencoders are feed-f
- Page 382 and 383: Depending upon the actual dimension
- Page 384 and 385: • __init__(): Here, you define al
- Page 386 and 387: Chapter 9And then we reshape the te
- Page 388 and 389: Chapter 9plt.imshow(x_test[index].r
- Page 390 and 391: Chapter 9Keeping the rest of the co
- Page 392 and 393: noise = np.random.normal(loc=0.5, s
- Page 394 and 395: Chapter 9x_train,validation_data=(x
- Page 396 and 397: Chapter 9import matplotlib.pyplot a
- Page 398 and 399: Chapter 9self.conv4 = Conv2D(1, 3,
- Page 400 and 401: Chapter 9You can see that the image
- Page 402 and 403: [ 367 ]Chapter 9Let us use the prec
- Page 404 and 405: Chapter 9Our autoencoder model take
- Page 406 and 407: We train the autoencoder for 20 epo
- Page 408 and 409: Chapter 90.97905576229095460.989323
- Page 410 and 411: Unsupervised LearningThis chapter d
- Page 412 and 413: Chapter 10Next we load the MNIST da
- Page 414 and 415: Chapter 10TensorFlow Embedding APIT
- Page 416 and 417: 3. Recompute the centroids using cu
- Page 418 and 419: Chapter 10Figure 4: Plot of the fin
- Page 422 and 423: [ 387 ]Chapter 10Colour mapping usi
- Page 424 and 425: Chapter 10# Calculating Neighbourho
- Page 426 and 427: We will also need to normalize the
- Page 428 and 429: Chapter 10ρρ(vv oo |h oo ) = σσ
- Page 430 and 431: # Generate the sample probabilityde
- Page 432 and 433: Chapter 10And the reconstructed ima
- Page 434 and 435: Chapter 10inpX = rbm.rbm_output(inp
- Page 436 and 437: Chapter 10(60000, 28, 28) (60000,)(
- Page 438 and 439: Chapter 10Figure 11: Summary of the
- Page 440 and 441: Chapter 10This chapter, along with
- Page 442 and 443: Reinforcement LearningThis chapter
- Page 444 and 445: Chapter 11And unlike unsupervised l
- Page 446 and 447: Chapter 11Normally, the value is de
- Page 448 and 449: Chapter 11• The next question tha
- Page 450 and 451: Chapter 11This neural network takes
- Page 452 and 453: Chapter 11The MuJoCo environment re
- Page 454 and 455: Chapter 11We will first import the
- Page 456 and 457: Chapter 11The αα is the learning
- Page 458 and 459: Chapter 11We set up the global valu
- Page 460 and 461: Chapter 11else:return np.argmax(sel
- Page 462 and 463: Chapter 11DQN to play a game of Ata
- Page 464 and 465: Chapter 11self.model.add( Conv2D(64
- Page 466 and 467: Chapter 11Here the action A was sel
- Page 468 and 469: Chapter 11Image source: https://arx
Chapter 10
In SOMs, neurons are usually placed at nodes of a (1D or 2D) lattice. Higher
dimensions are also possible but are rarely used in practice. Each neuron in the
lattice is connected to all the input units via a weight matrix. The following diagram
shows a SOM with 6 × 8 (48 neurons) and 5 inputs. For clarity, only the weight
vectors connecting all inputs to one neuron are shown. In this case, each neuron will
have seven elements, resulting in a combined weight matrix of size (40 × 5):
A SOM learns via competitive learning. It can be considered as a nonlinear
generalization of PCA and thus, like PCA, can be employed for dimensionality
reduction.
In order to implement SOM, let's first understand how it works. As a first step,
the weights of the network are initialized either to some random value or by taking
random samples from the input. Each neuron occupying a space in the lattice will be
assigned specific locations. Now as an input is presented, the neuron with the least
distance from the input is declared the winner (WTU). This is done by measuring the
distance between the weight vectors (W) and input vectors (X) of all neurons:
NN
dd jj = √∑(WW jjjj − XX ii ) 2
ii=1
Here, d j
is the distance of weights of neuron j from input X. The neuron with the
lowest d value is the winner.
Next, the weights of the winning neuron and its neighboring neurons are adjusted in
a manner to ensure that the same neuron is the winner if the same input is presented
next time.
[ 385 ]
Change language