Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Equation 10.7 - Data points' means over features (D)
inputs_mean = inputs.mean(axis=2).unsqueeze(2)
inputs_mean
Output
tensor([[[-0.3529],
[ 0.2426]],
[[ 0.9496],
[-1.3038]],
[[ 1.6489],
[ 3.6841]]])
As expected, six mean values, one for each data point. The unsqueeze() is there to
preserve the original dimensionality, thus making the result a tensor of (N, L, 1)
shape.
Next, we compute the biased standard deviations over the same dimension (D):
Equation 10.8 - Data points' standard deviations over features (D)
inputs_var = inputs.var(axis=2, unbiased=False).unsqueeze(2)
inputs_var
Layer Normalization | 823