pdfcoffee

The Math Behind Deep Learning
This chapter has presented the mathematical foundation behind the intuition. As
discussed, this topic is quite advanced and not necessarily required for practitioners.
However, it is recommended reading if you are interested in understanding what is
going on "under the hood" when you play with neural networks.
The next chapter will introduce the Tensor Processing Unit (TPU), a special chip
developed at Google for ultra-fast execution of many mathematical operations
described in this chapter.
References
1. Kelley, Henry J. (1960). Gradient theory of optimal flight paths. ARS Journal.
30 (10): 947–954. Bibcode:1960ARSJ...30.1127B. doi:10.2514/8.5282.
2. Dreyfus, Stuart (1962). The numerical solution of variational problems. Journal
of Mathematical Analysis and Applications. 5 (1): 30–45. doi:10.1016/0022-
247x(62)90004-5.
3. Werbos, P. (1974). Beyond Regression: New Tools for Prediction and Analysis in
the Behavioral Sciences. PhD thesis, Harvard University.
4. Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (1986-10-09).
Learning representations by back-propagating errors. Nature. 323 (6088): 533–536.
Bibcode:1986Natur.323..533R. doi:10.1038/323533a0.
5. LeCun, Y. (1987). Modèles Connexionnistes de l'apprentissage (Connectionist
Learning Models), Ph.D. thesis, Universite' P. et M. Curie, 1987
6. Herbert Robbins and Sutton Monro A Stochastic Approximation Method The
Annals of Mathematical Statistics, Vol. 22, No. 3. (Sep., 1951), pp. 400-407.
7. Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E. (June 2017). ImageNet
classification with deep convolutional neural networks (PDF). Communications of
the ACM. 60 (6): 84–90. doi:10.1145/3065386. ISSN 0001-0782.
8. From not working to neural networking. The Economist. 25 June 2016
[ 570 ]