Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Figure B.4 - From original to activated feature space
On the left, we have the original feature space, followed by the transformed feature
space in the center (corresponding to the output of the "Hidden Layer #0," before
the activation), and the activated feature space on the right.
Let’s focus on the right plot: As promised, the decision boundary is a straight line.
Now, pay attention to the grid lines there: They are twisted and turned beyond
recognition, as promised. This is the work of the activation function.
Moreover, I’ve plotted the decision boundary in the first two features spaces as
well: They are curves now!
It turns out, a curved decision boundary in the original feature
space corresponds to a straight line in the activated feature
space.
Cool, right? The first time I looked at those, many years ago, it was a defining
moment in my own understanding of the role and importance of activation
functions.
I showed you the trained model first to make an impact. At the beginning of the
training process, the visuals are not nearly as impressive.
336 | Bonus Chapter: Feature Space