Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

The minority class should have the largest weight, so each data
point belonging to it gets overrepresented to compensate for the
imbalance.
"But these weights do not sum up to one—isn’t it wrong?"
It is common to have weights summing up to one, sure, but this is not required by
PyTorch’s weighted sampler. We can get away with having weights inversely
proportional to the counts. In this sense, the sampler is very "forgiving." But it is
not without its own quirks, unfortunately.
It is not enough to provide a sequence of weights that correspond to each different
class in the training set. It requires a sequence containing the corresponding
weight for each and every data point in the training set. Even though this is a bit
annoying, it is not so hard to accomplish: We can use the labels as indexes of the
weights we computed above. It is probably easier to see it in code:
sample_weights = weights[y_train_tensor.squeeze().long()]
print(sample_weights.shape)
print(sample_weights[:10])
print(y_train_tensor[:10].squeeze())
Output
torch.Size([240])
tensor([0.0063, 0.0063, 0.0063, 0.0063, 0.0063, 0.0125, 0.0063,
0.0063, 0.0063, 0.0063])
tensor([1., 1., 1., 1., 1., 0., 1., 1., 1., 1.])
Since there are 240 images in our training set, we need 240 weights. We squeeze
our labels (y_train_tensor) to a single dimension and cast them to long type since
we want to use them as indices. The code above shows the first ten elements, so
you can actually see the correspondence between class and weight in the resulting
tensor.
The sequence of weights is the main argument used to create the
WeightedRandomSampler, but not the only one. Let’s take a look at its arguments:
Data Preparation | 291