pdfcoffee

Chapter 8
Epoch-#
Loss
BLEU Score
English
French
French
(Training)
(Test)
(true)
1 1.4119 1.957e-02 tom is special. tom est
special.
2 1.1067 2.244e-02 he hates
shopping.
il deteste faire
les courses.
(predicted)
elle est tres
bon.
il est tres
mineure.
3 0.9154 2.700e-02 did she say it? l a t elle dit? n est ce pas
clair?
4 0.7817 2.803e-02 i d rather
walk.
je prefererais
marcher.
5 0.6632 2.943e-02 i m in the car. je suis dans la
voiture.
...
245 0.0896 4.991e-02 she sued him. elle le
poursuivit en
justice.
246 0.0853 5.011e-02 she isn t poor. elle n est pas
pauvre.
247 0.0738 5.022e-02 which one is
mine?
248 0.1208 4.931e-02 i m getting
old.
249 0.0837 4.856e-02 t was worth a
try.
250 0.0967 4.869e-02 don t back
away.
lequel est le
mien?
je me fais
vieux.
ca valait
le coup d
essayer.
ne reculez pas!
je suis alle a
kyoto.
je suis toujours
inquiet.
elle l a
poursuivi en
justice.
elle n est pas
pauvre.
lequel est le
mien?
je me fais
vieux.
ca valait
le coup d
essayer.
ne reculez pas!
The full code for this example can be found in the source code accompanying this
chapter. You will need a GPU-based machine to run it, although you may be able
to run it on the CPU using smaller network dimensions (embedding_dim, encoder_
dim, decoder_dim), smaller hyperparameters (batch_size, num_epochs), and
smaller number of sentence pairs. To run the code in its entirety, run the following
command. The output will be written to console:
$ python seq2seq_wo_attn.py
In the next section, we will look at a mechanism to improve the performance of the
seq2seq network, by allowing it to focus on certain parts of the input more than on
others in a data-driven way. This mechanism is known as the Attention mechanism.
[ 327 ]