Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

It is hard to tell how badly our model is performing by looking at the loss only, if
we’re dealing with a "regression" problem like this. It is much better to visualize the
predictions.
Visualizing Predictions
Let’s plot the predicted coordinates and connect them using dashed lines, while
using solid lines to connect the actual coordinates. The first ten sequences of the
test set look like this:
fig = sequence_pred(sbs_seq, full_test, test_directions)
Figure 9.8 - Predictions
The results are, at the same time, very good and very bad. In half of the sequences,
the predicted coordinates are quite close to the actual ones. But, in the other half,
the predicted coordinates are overlapping with each other and close to the
midpoint between the actual coordinates.
Can We Do Better?
The encoder-decoder architecture is really interesting, but it has a bottleneck: The
whole source sequence gets to be represented by a single hidden state, the final
hidden state of the encoder part. Even for a very short source sequence like ours,
it’s quite a big ask to have the decoder generate the target sequence with so little
information.
"Can we make the decoder use more information?"
704 | Chapter 9 — Part I: Sequence-to-Sequence