Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Putting It All Together
In this chapter, we used the dataset of colored squares to, once again, predict the
coordinates of the last two corners (target sequence) given the coordinates of the
first two corners (source sequence). We built on top of our self-attention-based
encoder-decoder architecture, turning the former encoder and decoder classes
into "layers" and wrapping up its internal operations with "sub-layers" to add layer
normalization, dropout, and residual connections to each operation.
Data Preparation
The training set has the full sequences as features, while the test set has only the
source sequences as features:
Data Preparation
1 # Training set
2 points, directions = generate_sequences(n=256, seed=13)
3 full_train = torch.as_tensor(points).float()
4 target_train = full_train[:, 2:]
5 train_data = TensorDataset(full_train, target_train)
6 generator = torch.Generator()
7 train_loader = DataLoader(train_data, batch_size=16,
8 shuffle=True, generator=generator)
9 # Validation/Test Set
10 test_points, test_directions = generate_sequences(seed=19)
11 full_test = torch.as_tensor(test_points).float()
12 source_test = full_test[:, :2]
13 target_test = full_test[:, 2:]
14 test_data = TensorDataset(source_test, target_test)
15 test_loader = DataLoader(test_data, batch_size=16)
Model Assembly
Once again, we used the bottom-up approach to increasingly extend our encoderdecoder
architecture. Now, we’re revisiting the Transformer in a top-down
approach, starting from the encoder-decoder module (1). In its encode() and
decode() methods, it makes a call to an instance of the encoder (2), followed by a
call to an instance of the decoder (3). We’ll be representing the two called modules
(2 and 3) as boxes inside the box corresponding to the caller module (1).
Putting It All Together | 861