Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

The mask guarantees that the predicted x 2 (in the first step)
won’t change the predicted x 2 (in the second step), because
predictions are made based on past data points only.
trg_masks = subsequent_mask(2)
out = decself(inputs, trg_masks)
out
Output
tensor([[[0.4137, 0.3727],
[0.4132, 0.3728]]], grad_fn=<AddBackward0>)
These are the predicted coordinates of both x 2 and x 3 . They are very close to each
other, but that’s just because we’re using an untrained model to illustrate the
mechanics of using target masks for prediction. The last prediction is, once again,
concatenated to the previous "query."
inputs = torch.cat([inputs, out[:, -1:, :]], dim=-2)
inputs
Output
tensor([[[-1.0000, 1.0000],
[ 0.4132, 0.3728],
[ 0.4132, 0.3728]]], grad_fn=<CatBackward>)
But, since we’re actually done with the predictions (the desired target sequence has
a length of two), we simply exclude the first data point in the query (the one coming
from the source sequence), and are left with the predicted target sequence:
inputs[:, 1:]
756 | Chapter 9 — Part II: Sequence-to-Sequence